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ABSTRACT 

This  work  focusesed  on  the  mathematical  foundations  of  nonlinear  systems  analysis  and 
feedback  control.  It  addressed  the  continuing  development  of  systems  analysis  tools  based  on 
input  to  state  stability  and  related  notions  of  detectability,  regulation,  and  stabilization,  as 
well  as  the  study  of  new  theoretical  questions  arising  from  the  study  of  biomolecular  cellular 
mprhanisms,  seen  as  a  source  of  inspiration  for  novel  sensor,  actuation,  and  control  architectures. 
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1  Overview 

This  research  addresses  the  mathematical  foundations  of  nonlinear  systems  analysis  and  feedback 
control.  Nonlinear  control  theory  is  of  central  importance  to  the  Air  Force  mission,  and  the 
study  of  the  associated  mathematical  theory  is  of  undisputed  relevance.  The  control  of  highly 
nonhnear  systems  broadly  impacts  apphcations  ranging  from  robust  feedback  controllers  for 
advanced  high-performance  aircraft,  and  reconfigurable  flight  control  systems,  to  the  design  of 
autonomous  aerial  vehicles  and  of  smart-munitions  guidance  systems. 

The  design  and  analysis  of  nonlinear  feedback  systems  has  recently  undergone  an  excep¬ 
tionally  rich  period  of  progress  and  maturation,  fueled,  to  a  great  extent,  by  (1)  the  discovery 
of  certain  basic  conceptual  notions,  and  (2)  the  identification  of  classes  of  systems  for  which 
systematic  decomposition  approaches  can  result  in  effective  and  easily  computable  control  laws. 
These  two  aspects  are  complementary,  since  the  latter  approaches  are,  typically,  based  upon  the 
inductive  verification  of  the  validity  of  the  former  system  properties  under  compositions  (the 
“activation”  of  theoretical  concepts  leads  to  “constructive”  control).  It  is  perhaps  in  the  first 
of  these  aspects,  and  in  particular  in  the  precise  formulation  of  questions  of  robustness  with 
respect  to  disturbances,  and  of  stabilization  conditions,  formulated  in  the  paradigm  of  input  to 
state  stability,  that  the  Pi’s  previous  work  has  had  the  most  influence,  although  our  research 
covers  a  wide  spectrum  of  other  subjects  as  well.  A  large  part  of  the  project  deals  with  such 
issues. 

One  relatively  recent  direction  of  the  Pi’s  research  is  the  study  of  biological,  and  in  par¬ 
ticular  intra-cellular,  control  and  signal  processing  mechanisms.  As  a  theoretician,  the  PI  is 
particularly  interested  in  new  mathematical  questions  in  systems  and  control  theory  that  arise 
when  analyzing  molecular  biology  models.  Evolution  has  resulted  in  robust,  highly  nonhnear, 
and  hybrid  feedback  systems.  Recent  advances  in  genomic  research  are  continuously  adding 
detailed  knowledge  of  such  systems’  architecture  and  operation,  and  one  may  reasonably  argue 
that  they  will  constitute  a  rich  source  of  inspiration  for  innovative  solutions  to  problems  of 
control  and  commimication  engineering,  as  well  as  sensor  and  actuator  design  and  integration. 
Thus,  another  major  component  of  this  project  deals  with  problems  of  biological  control. 

The  emphasis  of  this  project  is  on  the  development  of  basic  principles,  and  on  the  com¬ 
munication  of  these  results  to  those  in  the  engineering  community  (including  especially  other 
AFOSR  researchers)  who,  in  turn,  employ  these  techniques  in  apphcations. 

1.1  Input  to  State  Stability  cind  Related  Notions 

We  discuss  general  finite-dimensional  systems  x{t)  =  f(x{t),u{t)),  y{t)  =  h{x{t)),  in  the  sense 
of  nonhnear  control  under  standard  regularity  assumptions  (see  Section  2.1  for  more  details). 
There  are  two  conceptually  very  different  ways  to  formulate  the  notion  of  stability  of  such 
systems. 

One  of  them,  the  purely  input/output  approach,  relies  on  operator-theoretic  techniques. 
Among  the  main  contributions  to  this  area,  one  may  cite  the  foundational  work  by  Zames, 
Sandberg,  Desoer,  Safanov,  Vidyasagar,  and  others.  In  this  approach,  a  “system”  is  a  causal 
operator  F  =  Fx(p)  :  u{-)  y(-)  between  spaces  of  signals  (for  fixed  initial  states),  and  “stabil¬ 

ity”  is  taken  to  mean  that  F  maps  bounded  inputs  into  bounded  outputs,  or  finite-energy  inputs 
into  finite-energy  outputs.  More  stringent  typical  requirements  in  this  context  are  that  the  gain 
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of  F  be  finite  (in  more  classical  mathematical  terms,  that  the  operator  be  bounded),  or  that  it 
have  finite  incremental  gain  (mathematically,  that  it  be  globally  Lipschitz).  The  input /output 
approach  has  been  extremely  successful  in  the  robustness  analysis  of  linear  systems  subject  to 
nonlinear  feedback  and  mild  nonlinear  uncertainties,  and  in  general  in  the  area  that  revolves 
around  the  various  versions  of  the  small-gain  theorem.  Moreover,  geometric  characterizations 
of  robustness  (gap  metric  and  the  like)  are  elegantly  carried  out  in  this  framework.  Finally,  i/o 
stability  provides  a  natural  setting  in  which  to  study  the  classification  and  parameterization  of 
dynamic  controllers. 

On  the  other  hand,  there  is  the  model-based,  or  state-space  approach  to  systems  and  sta¬ 
bility.  In  this  approach,  there  is  a  standard  notion  of  stability,  namely  Lyapunov  asymptotic 
stability  of  the  unforced  system.  Associated  to  such  a  system,  there  is  the  above-mentioned 
operator  F  mapping  inputs  (forcing  functions)  into  state  trajectories  (or  into  outputs,  if  par¬ 
tial  measurements  on  states  are  of  interest).  It  becomes  of  interest  then  to  ask  to  what  extent 
Lyapunov-like  stability  notions  for  a  state-space  system  are  related  to  the  stability,  in  the  senses 
discussed  in  the  previous  paragraph,  of  the  associated  operator  F.  It  is  well-known  that,  in  con¬ 
trast  to  the  case  of  (finite-dimensional)  linear  systems,  where  there  is  -subject  to  mild  technical 
assumptions-  an  equivalence  between  state-space  and  i/o  stability,  for  nonhnear  systems  the 
two  types  of  properties  are  not  so  closely  related.  Even  for  the  very  special  and  comparatively 
simple  case  of  “feedback  linearizable”  systems,  this  relation  is  far  more  subtle  than  it  might 
appear  at  first  sight:  if  one  first  linearizes  a  system  and  then  stabilizes  the  equivalent  lineariza¬ 
tion,  in  terms  of  the  original  system  one  does  not  in  general  obtain  a  closed-loop  system  that 
is  input /output  stable  in  any  reasonable  sense. 

This  leads  one  to  focus  on  the  study  of  the  dependence  of  state  trajectories  on  the  size  of 
inputs,  a  study  which  is  especially  relevant  when  the  inputs  in  question  represent  disturbances 
acting  on  a  system.  For  not  necessarily  linear  systems,  there  are  various  possible  formulations  of 
system  stability  with  respect  to  input  perturbations.  (For  linear  systems,  similar  considerations 
led  to  the  development  of  gains  and  the  operator-theoretic  approach,  including  the  formulation, 
when  using  norms,  of  control.)  One  candidate  for  such  a  formulation  is  the  property 
called  “input  to  state  stability”  (ISS),  introduced  by  the  PI  in  1989.  In  very  informal  terms,  the 
ISS  property  translates  into  the  statement  that  “no  matter  what  is  the  initial  state,  if  the  inputs 
are  small,  then  the  state  must  eventually  be  small,  and  the  overshoot  depends  on  the  size  of  the 
initial  state.”  The  ISS  notion  can  be  stated  in  several  equivalent  manners,  which  indicates  that 
it  is  a  mathematically  natural  concept:  dissipation,  robustness  margins,  or  asymptotic  gains. 

The  three  most  basic  notions  of  stability  in  the  theory  of  finite-dimensional  linear  systems 
are  (1)  internal  stability  (behavior  of  states),  (2)  detectability  (external  behavior  determines 
how  states  evolve),  and  (3)  input/output  stability  (transfer  function  has  no  poles  in  the  right- 
hand  plane).  These  generalize,  respectively,  to  (1)  ISS,  (2)  lOSS  (input  and  output  to  state 
stability),  and  (3)  lOS  (input  to  output  stability),  for  general  finite-dimensional  nonlinear 
systems.  The  observation  that  a  system  is  internally  stable  if  and  only  if  it  is  i/ o  stable  and 
detectable  generalizes  to  “ISS  equals  lOSS  and  lOS”  in  an  immediate  fashion,  providing  a 
conceptual  rmity  to  the  field. 

Input  to  state  stability  proved  to  be  a  very  useful  paradigm  in  the  study  of  nonlinear  stability 
for  systems  subject  to  external  effects,  especially  in  connection  to  issues  of  robust  and  adaptive 
control,  stochastic  and  hybrid  versions,  or  in  its  variants  for  detectability  and  output  stability 
(loss,  lOS)  and  for  integral  stability  (iISS). 
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The  ISS  approach  differs  fundamentally  from  the  operator-theoretic  ones,  first  of  all  because 
it  takes  account  of  initial  states  in  a  manner  fully  compatible  with  Lyapunov  stability.  Second, 
boundedness  (finite  gain)  is  far  too  strong  a  requirement  for  general  nonlinear  operators,  and 
it  must  be  replaced  by  ‘‘nonlinear  gain  estimates,”  in  which  the  norms  of  output  signals  are 
bounded  by  a  nonlinear  function  of  the  norms  of  inputs;  the  definition  of  ISS  incorporates 
such  gains  in  a  natural  way.  Yet  another  way  in  which  the  approach  differs  from  operator- 
theoretic  quantifications  of  stability  is  that  one  does  not  focus  upon  the  actual  numerical  values 
of  gains  (operator  norms)  but,  rather,  on  the  more  qualitative  question  of  existence  ofestimates; 
for  (finite-dimensional)  linear  systems,  of  course,  all  these  questions  are  equivalent,  and  they 
amount  to  the  requirement  that  the  origin  should  be  asymptotically  stable  with  respect  to  the 
unforced  dynamics,  but  not  so  for  general  nonlinear  systems.  One  may  say  that  the  ISS  view 
is  “topological”  in  contrast  to  the  more  “metric”  view  of  standard  operator  approaches. 

Recent  Accomplishments 

Next,  we  highlight  next  some  of  our  recent  accomplishments  in  the  ISS  area;  some  more  details 
on  several  of  these  items  are  provided  in  Section  2.1. 

We  were  able  to  complete  the  dissipation  characterizations  of  input  to  output  stability  (lOS), 
of  input-output  to  state  stability,  and  of  the  integral  ISS  (iISS)  property  and  its  semi-global 
versions.  ([s63],[s64],[s52]).  Asymptotic-gain  equivalences  were  found  for  lOS,  cf.  [s75],  and 
loss,  cf.  [s74];  see  [sl6]  for  the  most  general  results  along  these  lines. 

Proving  these  asymptotic  characterizations  requires  the  generalization  of  the  Filippov- 
Wazewski  Relaxation  Theorem  to  infinite  time  intervals,  a  result  which  is  of  independent  interest 
in  the  field  of  differential  inclusions,  see  [s35]  for  the  finite-dimensional  case  and  [s56]  for  the 
corresponding  theorem  in  Banach  spaces.  This  led,  in  turn,  to  a  new  theorem  showing  that, 
for  locally  Lipschitz  differential  inclusions,  uniform  and  non-uniform  global  asymptotic  stability 
are  equivalent,  cf.  [s4]. 

In  applications  such  as  adaptive  control,  often  suitable  tracking  behavior  is  obtained  only 
under  assumptions  of  slow  variation  of  inputs;  this  gave  rise  to  the  study  of  ISS  with  respect  to 
input  derivatives;  the  papers  [s68],  [s43]  describe  relations  and  differences  with  the  ISS  notion, 
and  applications. 

Measurement  to  error  stability  (MES)  is  a  notion  of  partial  detectability  for  nonlinear  sys¬ 
tems,  which  unifies  the  many  concepts  associated  to  ISS  and  which  plays  a  key  role  in  regulator 
theory.  This  notion  is  still  poorly  understood,  but  some  progress  was  reported  in  [s58]. 

For  nonlinear  cascade  systems  resulting  from  input-output  linearization,  strong  assumptions 
have  to  be  made  in  order  to  obtain  stability,  typically  that  the  driven  system  is  ISS.  The 
papers  [s73],  [s55]  dealt  with  the  case  when  the  driven  subsystem  is  merely  integral  ISS,  and 
characterized  the  admissible  integral  ISS  gains  for  stability.  The  result  was  used  to  develop 
a  new  observer-based  backstepping  design  under  suitable  growth  assumptions  on  nonlinear 
damping  terms. 

The  papers  [s67],  [s50]  introduced  and  studied  the  notion  of  output-input  stability  (OIS), 
which  was  motivated  by  and  relates  to  the  minimum-phase  property.  The  definition  requires 
the  state  and  input  to  be  bounded  by  a  suitable  function  of  the  output  and  derivatives  of  the 
output,  modulo  a  decaying  term  depending  on  initial  conditions,  and  includes  affine  systems  in 
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global  normal  form  whose  internal  dynamics  are  ISS.  An  extension  to  nonlinear  systems  of  a 
basic  result  from  linear  adaptive  control,  using  this  notion,  was  shown  there  as  well. 

A  very  abstract  and  formal  ISS-like  small  gain  theorem,  in  a  form  applicable  to  incremental 
stability,  detectability,  and  input/output  maps  (possibly  nonrealizable  by  finite-dimensional 
S3rstems)  was  given  in  [s49]. 

In  [s40],  we  gave  an  example  of  a  globally  asymptotically  stable  time-invariant  system 
which  can  be  destabilized  by  some  integrable  perturbation;  this  answered  (in  the  negative)  an 
open  problem  posed  by  Laurent  Praly  concerning  the  existence  of  continuously  differentiable 
Lyapunov  functions  with  globally  bounded  gradients. 

The  paper  [s51]  dealt  with  the  design  of  state  feedback  control  laws  that  render  a  closed-loop 
system  iISS  with  respect  to  disturbances,  using  an  appropriate  concept  of  control  Lyapunov 
function  (ilSS-CLF),  whose  existence  is  shown  to  lead  to  an  explicit  construction  of  such  a 
control  law. 

In  [s38],  [Sl4],  we  returned  to  a  question  first  addressed  in  our  original  ISS  paper  in  1989.  For 
systems  admitting  a  continuous  stabilizer,  we  had  shown  that  it  is  always  possible  to  “robustify” 
the  given  control  law  so  as  to  obtain  a  new  feedback  which  is  ISS  with  respect  to  actuator  errors. 
Regularity  of  the  feedback  is  essential,  and  for  several  years  it  was  an  open  question  to  obtain 
a  version  of  this  result  for  systems  which  are  only  stabilizable  by  discontinuous  feedback  (for 
instance,  nonholonomic  mechanical  systems).  The  breakthrough  came  with  the  realization  that 
semiconcave  control  Lyapunov  functions  provide  the  key  missing  ingredient,  and  now  we  were 
able  to  obtain  the  general  result,  expressed  in  both  the  sampling  and  Euler  solution  senses. 

The  use  of  switching  strategies  for  adaptive  control  requires  the  development  of  “LaSalle- 
like”  invariance  principles,  which  lead  in  turn  to  our  study  of  ISS-like  characterizations  of 
observability  of  nonlinear  systems,  see  [s59]. 

The  goal  of  the  Pi’s  work  in  this  area  is  highly  ambitious:  the  complete  reformulation 
of  the  foundations  of  nonlinear  control  based  on  ISS-like  ideas.  This  goal  is  very  long-term, 
but  definitely  worth  the  effort:  the  payoff  will  be  in  the  development  of  a  consistent,  elegant, 
and  ultimately  design-oriented,  systematic  approach  to  the  subject.  Previous  grants  supjmrted 
research  which  led  to  truly  exciting  advances  during  the  last  few  years.  However,  the  story  is 
by  no  means  complete,  and  we  hope  to  be  able  to  continue  developing  the  theory. 

1.2  Systems  and  Control  Problems  Inspired  by  Molecular  Biology 

Research  in  molecular  biology  and  genomics  has  provided,  and  will  continue  to  produce,  a 
wealth  of  data  describing  the  ingredients  of  cell  behavior.  There  is  a  clear  need  to  integrate  this 
knowledge,  as  a  prerequisite  to  a  deeper  scientific  understanding  as  well  as  medical  applications. 
Indeed,  the  new  field  of  “systems  biology”  has  arisen  (see  e.g.  the  new  lEE  journal  Systems 
Biology)  having  as  its  goal  the  imderstanding  of  the  basic  dynamic  processes,  feedback  control 
loops,  and  signal  processing  mechanisms  underlying  life.  Conversely,  since  evolution  has  resulted 
in  robust,  highly  nonlinear  and  hybrid  feedback  systems,  it  is  reasonable  to  expect  that  the  study 
of  systems  biology  will  constitute  a  rich  source  of  inspiration  for  innovative  solutions  to  control 
and  communication  engineering  problems,  as  well  as  sensor  and  actuator  design  and  integration. 
The  PI  has  recently  started  the  study  of  several  mathematical  questions  in  systems  and  control 
theory  which  arise  from  molecular  biology  models. 
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The  need  for  mathematical  models  of  feedback  in  cellular  biology  has  been  long  recognized, 
and  seminal  work  was  done  during  the  past  30  or  more  years  by  researchers  such  as  Glass, 
Goldbeter,  Kaufl&nan,  Othmer,  Savageau,  Segel,  Tyson,  and  many  others.  What  makes  the 
present  time  special  is  the  availability  of  huge  amounts  of  data  -  generated  by  the  genomics 
and  proteomics  projects,  and  research  efforts  dealing  with  the  characterization  of  signaling 
networks  -  as  well  as  the  possibility  for  experimental  design  afforded  by  genetic  engineering 
tools  (gene  knock-outs  and  insertions,  PCR)  and  measurement  technology  (Green  Fluorescent 
Protein  and  other  reporters,  as  well  as  gene  arrays). 


Background 

The  life  sciences  are  in  the  midst  of  a  major  revolution,  which  will  have  fundamental  implications 
in  biological  knowledge  and  medicine.Work  in  genomics  has  as  its  objective  the  complete  decod¬ 
ing  of  DNA  sequences,  providing  what  one  may  call  a  “parts  list”  for  the  proteins  potentially 
present  in  every  cell  of  the  organism  being  studied.  The  elucidation  of  the  three-dimensional 
structure  of  the  proteins  so  described  is  the  goal  of  the  area  of  proteomics.  The  shape  of  a 
protein,  in  turn,  determines  its  function:  proteins  interact  with  each  other  through  “lego-like” 
fitting  of  parts  in  “lock  and  key”  fashion,  and  their  conformation  also  enhances  or  represses 
DNA  expression  through  selective  binding.  One  of  the  main  themes  and  challenges  in  current 
molecular  biology  lies  in  the  understanding  of  cell  behavior  in  terms  of  cascades  and  feedback 
interconnections  of  elementary  “modules”  which  appear  repeatedly.  On  the  other  hand,  the 
successes  of  systems  theory  have  been,  in  large  part,  due  precisely  to  its  ability  to  analyze 
complicated  structiures  on  the  basis  of  the  behavior  of  elementary  subsystems,  each  of  which 
is  “nice”  in  a  suitable  input/output  sense  (stable,  passive,  etc),  in  conjunction  with  the  use  of 
tools  such  as  small  gain  theorems.  Thus,  control-oriented  modeling  and  analysis  of  feedback 
interconnections  will  become  an  integral  component  of  building  effective  models  of  biological 
S)^tems. 

One  may  view  cell  life  as  a  huge  “wireless  network”  of  interactions  among  proteins,  DNA, 
and  smaller  molecules  involved  in  signaling  and  energy  transfer.  As  a  large  system,  the  external 
inputs  to  a  cell  include  physical  (UV  radiation,  temperature)  as  well  as  chemical  (drugs,  hor¬ 
mones,  nutrients)  signals.  Its  outputs  include  chemicals  which  may  in  turn  affect  other  cells, 
the  movement  of  flagella  or  pseudopods,  the  activation  of  transcription  factors,  and  so  forth. 
The  study  of  cell  networks  leads  to  the  formulation  of  a  large  number  of  questions  of  a  systems 
theory  flavor.  What  is  special  about  the  information-processing  capabilities,  or  input/output 
behaviors,  of  such  networks,  and  how  does  one  characterize  these  behaviors  in  terms  familiar 
to  control  theory?  What  “system-theoretic  modules”  appear  repeatedly?  What  are  the  sta¬ 
bility  properties  of  the  various  cascades  and  feedback  loops  which  appear  in  cellular  signaling 
networks?  Inverse  or  “reverse  engineering”  issues  include  the  estimation  of  system  parameters 
(such  as  reaction  constants)  as  well  as  estimation  of  state  variables  (concentration  of  protein, 
RNA,  and  other  chemical  substances)  from  input/output  experiments.  Generically,  these  ques¬ 
tions  may  be  viewed  respectively  as  tlie  identification  and  observer  (or  filtering)  problems  which 
are  at  the  center  of  much  of  control  theory. 
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Recent  Accomplishments 

Control  and  systems  problems  motivated  by  molecular  biology  tend  to  resemble  similar  prob¬ 
lems  that  arise  in  engineering  control  or  dynamical  systems.  However,  these  resemblances  are 
oftentimes  misleading,  because  the  precise  problem  formulations  may  be  very  different  from 
those  in  the  respective  more  classical  applications.  This  issue  has  driven  much  of  the  Pi’s 
interest  in  the  subject. 

There  follows  a  quick  overview  of  several  areas  of  the  Pi’s  recent  research.  More  details  on 
selected  topics  are  provided  in  Section  2.2. 

Monotone  Control  Systems 

Mitogen-activated  protein  kinase  (MARK)  cascades  represent  a  “biological  module”  or  sub¬ 
circuit  which  is  ubiquitous  in  eukaryotic  cell  signal  transduction  processes  and  is  a  critical 
component  of  pathways  involved  in  cell  proliferation,  differentiation,  movement,  and  death. 
Appearing  in  several  variants,  this  system  is  made  up  of  a  cascade  of  three  smaller  subsystems, 
with  certain  feedback  loops  acting  in  regulatory  modes.  Research  into  MAPK  cascade  dynamics 
is  a  major  area  of  interest  in  biology,  with  implications  not  only  to  natural  processes  but  even 
to  areas  such  as  bio-terrorism  (Bacillus  anthracis,  which  causes  of  Anthrax,  acts  by  secreting  a 
toxin  called  Lethal  Factor  (LF),  which  disrupts  the  “connection”  between  the  second  and  third 
subsystems  in  the  cascade). 

Our  research  in  this  area  was  originally  motivated  by  work  of  Kholodenko,  which  dealt  with 
the  possible  onset  of  oscillations  under  negative  feedback  in  a  MAPK  cascade,  specifically,  the 
inhibitory  phosphorylation  of  upstream  SOS  by  p42/p44  MAPK  (ERK).  Since  such  periodic 
behavior  has  not  been  observed  experimentally,  one  would  like  to  understand  what  conditions 
guarantee  the  non-existence  of  oscillations  in  systems  of  this  kind  (modeled  mathematically  by 
several  authors. 

Small-gain  theorems  are  routinely  used  in  control  theory  in  order  to  guarantee  stability. 
However,  classical  small-gain  theorems  cannot  be  used,  at  least  in  any  obvious  “off  the  shelf” 
fashion,  if  the  location  of  the  closed-loop  steady-states  depends  on  the  precise  gains  of  the 
feedback  law  (or  if  there  are  multiple  such  states).  Negative  (or  “inhibitory”)  feedback  in 
molecular  biology  is  almost  never  of  the  form  u  —  — fc(x  —  x*),  which  would  preserve  the 
equilibrium  x*.  Rather,  it  may  take  a  form  such  as  “l/(fc  -i-  a;)”  so  that  the  closed-loop  steady 
state  depends  on  the  actual  value  of  the  parameter  k.  For  example,  the  equilibrium  x  =  4  in 
i:  =  -x  +  4  gets  moved  to  x  =  1  under  the  inhibitory  feedback  resulting  in  x  =  -x  +  4/(3  +  x). 
In  [s61]  and  [s60],  we  introduced  the  notions  of  asymptotic  amplitude  for  signals,  and  associated 
Cauchy  gains  for  input /output  systems,  and  provided  a  Lyapunov-like  characterization  which 
allows  the  estimation  of  gains  for  state-space  systems.  We  then  stated  a  small-gain  theorem 
expressed  in  terms  of  Cauchy  gains,  and  used  these  results  to  obtain  a  very  tight  estimate  of 
the  onset  of  Hopf  bifurcations  in  the  MAPK  model  studied  by  Kholodenko. 

The  results  on  Cauchy  gains  allowed  us  to  deal  with  a  restricted  type  of  MAPK  cascade 
(single-phosphorylation  at  each  stage).  However,  a  fuller,  and  more  realistic,  model  was  harder 
to  analyze.  The  breakthrough  came  when  we  realized  (with  David  Angeli)  that  each  subsystem 
in  the  cascade  is  a  monotone  system  with  inputs^  with  respect  to  an  appropriate  partial  order 
in  states. 

Monotone  systems  are  dynamical  systems  x  =  /(x)  for  which  trajectories  preserve  a  partial 
ordering  in  They  include  the  subclass  of  cooperative  systems,  for  which  different  state 
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variables  reinforce  each  other  (positive  feedback)  as  well  as  certain  more  general  systems  in 
which  each  pair  of  variables  may  affect  each  other  in  either  positive  or  negative,  or  even  mixed, 
forms.  Among  the  classical  references  in  this  area  are  the  textbook  by  Hal  Smith  by  Hirsh 
and  Smale.  The  concept  of  monotone  system  had  been  traditionally  defined  only  for  systems 
with  no  external  inputs  (nor  outputs).  The  first  objective  of  the  papers  [s57]  and  [s31]  was  to 
extend  the  notion  of  monotone  systems  to  systems  with  inputs  and  outputs,  and  then  to  provide 
easily  verifiable  infinitesimal  characterizations  of  monotonicity  (expressed  in  nonsmooth  analysis 
terms,  via  Bouligand  tangent  cones).  This  is  by  no  means  a  purely  academic  exercise,  but  it  is 
a  necessary  first  step  in  the  study  of  interconnections,  especially  including  feedback  loops,  built 
up  out  of  monotone  components. 

We  also  introduced  the  notion  of  steady-state  response  for  every  constant  input,  or  static 
input /output  characteristic,  and  showed,  in  particular,  that  such  responses  are  always  well- 
defined  for  the  basic  MAPK  subsystems,  no  matter  what  are  the  form  of  the  kinetics  or  the 
numerical  values  of  parameters.  (This  fact  requires  a  careful  proof  -even  if  biologists  always 
assume  it  to  be  true-  as  it  amounts  to  proving  a  global  stability  result  for  a  nonlinear  system.) 
Cascades  of  monotone  systems  are  easily  shown  to  be  monotone,  and  steady-state  responses 
also  behave  well  under  composition,  but  negative  feedback  typically  destroys  the  monotonicity 
of  the  original  flow,  and  also  potentially  destabihzes  the  resulting  closed-loop  sjrstem.  The 
main  result  in  [s57]  and  [s31]  was  a  small-gain  theorem  for  negative  feedback  loops  involving 
monotone  sjrstems  with  well-defined  i/o  characteristics,  and  applied,  in  particular,  to  MAPK 
cascades. 

Although  motivated  by  MAPK  cascades,  the  results  are  far  more  general.  The  paper  [sl3] 
presented  a  variant  of  the  small-gain  theorem  firom  [s57]  and  [s31],  suitable  for  “almost  global” 
(meaning  that  the  dommn  of  attraction  is  open  dense)  stability  of  monotone  control  S3rstems 
which  have  well-defined  “almost”  i/o  characteristics.  This  variant  was  required  by  the  study 
in  [s37]  and  [s34],  which  presented  small-gain  theorems  guaranteeing  the  lack  of  oscillatory 
or  more  complicated  behavior  in  a  large  class  of  Lotka-Volterra  systems  with  predator-prey 
interactions. 

Yet  another  application  of  these  ideas  can  be  foimd  in  the  paper  [s36],  dealing  with 
diemostats,  which  describe  the  interaction  between  microbial  species  which  are  competing  for 
a  single  nutrient.  For  chemostats,  a  well-known  fact  is  the  so-called  “competitive  exclusion 
principle,”  which  states  roughly  that  in  the  long  run  only  one  of  the  species  survives.  This  is  in 
contrast  to  what  is  observed  in  nature,  where  several  species  seem  to  coexist.  This  discrepancy 
has  lead  many  researchers  to  propose  modifications  to  the  model  that  bring  theory  and  practice 
in  better  accordance.  In  particular,  [s36]  added  death  rates  due  to  crowding  effects,  and  for 
such  modified  systems,  presented  an  easily  checkable  condition  on  coefiicients  which  guarantees 
the  existence  of  a  global  attractor. 

A  very  different  line  of  research  deals  with  positive  feedback  loops.  Starting  with  systems 
which  have  a  well-defined  i/o  characteristic  and  are  also  monotone,  positive  feedback  preserves 
monotonicity  but,  in  general,  introduces  multiple  steady  states.  Multi-stationarity  by  positive 
feedback  is  a  mechanism  that  has  been  long  proposed  as  a  molecular-biological  basis  for  cell 
differentiation,  development,  and  periodic  behavior  described  by  relaxation  oscillations,  since 
the  classic  work  by  Delbriick,  who  suggested  in  1948  that  multi-stability  could  explain  cell 
differentiation,  and  continuing  to  the  present.  Using  the  theory  of  strictly  monotone  systems, 
together  with  basic  facts  about  system  gains,  we  were  able  in  [s3]  and  [s47]  to  show  that  the 
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location  and,  more  importantly,  stability  properties  of  steady  states,  can  be  determined  easily 
from  a  planar  plot,  and  a  theorem  guarantees  that  every  trajectory,  except  at  most  for  a  set  of 
measure  zero  of  initial  conditions,  converges  to  one  of  the  steady  states  so  identified.  We  also 
gave  a  discussion  of  hysteresis  behavior,  as  well  as  a  subtle  counterexample  showing  that  mono¬ 
tonicity  plays  a  crucial  role,  and  cannot  be  dispensed  with  as  an  assumption.  The  paper  [s2] 
deals  with  the  application  to  biological  models,  as  well  as  graphical  tests  for  monotonicity. 

Systems  Identification  and  Reverse  Engineering 

The  paper  [s48]  proposed  a  novel  quantitative  method  for  determining  functional  interac¬ 
tions  in  cellular  signaling  and  gene  networks.  The  method  can  be  used  to  explore  cell  systems 
at  a  mechanistic  level,  or  applied  within  a  modular  framework,  which  dramatically  decreases 
the  number  of  variables  to  be  assayed.  The  topology  and  strength  of  network  coimections  are 
retrieved  from  experimentally  measured  network  responses  to  successive  perturbations  of  all 
modules.  In  addition,  the  method  can  reveal  functional  interactions  even  when  the  compo¬ 
nents  of  the  system  are  not  all  known,  in  which  case  some  connections  retrieved  by  the  anal3rsis 
will  not  be  direct  but  correspond  to  the  interaction  routes  through  unidentified  elements.  The 
method  was  tested  and  illustrated  using  computer-generated  responses  of  a  modeled  MAPK 
cascade  and  gene  network. 

The  question  of  determining  such  interactions  fits  in  the  general  framework  of  realization 
and  identification  theory,  but  is  technically  different  because  of  the  lack  of  closure  under  con¬ 
catenation  of  biologically  realistic  inputs.  (Thus,  technically,  the  problem  is  no  more  one  of 
semigroup  representation,  as  classically  done  in  control  theory.)  A  related  general  question 
is:  given  a  set  of  differential  equations  whose  description  involves  vmknown  parameters,  such 
as  reaction  constants  in  chemical  kinetics,  and  supposing  that  one  may  at  any  time  measure 
the  values  of  some  of  the  variables  and  possibly  choose  external  inputs  from  a  finitely  param¬ 
eterized  class  to  help  excite  the  system  (but  without  assuming  any  semigroup  closure  of  this 
class  of  inputs),  how  many  experiments  are  sufficient  in  order  to  obtain  all  the  information 
that  is  potentially  available  about  the  parameters?  The  paper  [s53]  established  that  the  best 
possible  answer  (assuming  exact  measurements)  is  2r  -I- 1  experiments,  where  r  is  the  number 
of  parameters.  Moreover,  in  a  precise  mathematical  sense,  a  generic  set  of  such  experiments 
suffices. 

Finally,  we  continued  to  work  on  systems  identification  regarded  as  an  instance  of  the  general 
problem  of  “learning”  an  imknown  function.  The  papers  [s9]  and  [s70]  took  a  computational 
learning  theory  approach  to  a  problem  of  linear  systems  identification:  assume  that  inputs  are 
generated  randomly  from  a  known  class  of  finite  bandwith,  specifically  that  they  are  (unknown) 
linear  combinations  of  k  known  sinusoidals,  and  that  the  output  of  the  system  is  observed  at 
some  single  instant  of  time.  The  main  result  establishes  that  the  number  of  samples  needed 
for  identification  with  small  error  and  high  probability,  independently  from  the  distribution  of 
inputs,  scales  polynomially  with  n,  the  system  dimension,  and  logarithmically  with  k. 

Dynamics  and  Observers  for  Biochemical  Networks 

Among  other  tasks,  the  immune  system  is  charged  with  the  destruction  and  elimination 
of  invading  organisms  and  of  the  toxic  products  that  they  produce,  as  well  as  the  destruction 
of  virus-infected  or  mutated  cells.  One  of  the  most  challenging  problems  in  the  study  of  the 
immune  system  is  to  understand  how  it  manages  to  distinguish  among  “self”  and  “other”  while 
still  being  able  to  react  fast. 
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One  approach  to  this  question  was  proposed  by  McKeithan,  who  suggested  that  a  chain  of 
modifications  of  the  T^cell  receptor  complex,  via  tyrosine  phosphorylation  and  other  reactions, 
may  give  rise  to  both  increased  sensitivity  and  selectivity  of  response.  This  process,  which  he 
called  ‘‘kinetic  proofreading”  because  of  its  analogy  to  an  older  model  proposed  by  Hopfield  for 
DNA  error  correction,  was  modeled  by  McKeithan  for  T-cell  receptor  (TCR)  and  peptide-major 
histocompatibility  complex  (MHC)  interactions  by  means  of  a  certain  set  of  nonlinear  ordinary 
differential  equations.  The  steady  states  that  result  are  interpreted  as  a  “signal”  for  antigen 
recognition.  Thus,  it  was  natural  to  ask  if  the  steady  states  of  this  system  are  unique,  if  they 
are  asymptotically  stable,  and  so  forth. 

This  led  the  PI  to  a  new  direction  of  research.  The  paper  [s66]  dealt  with  the  theory  of 
structure,  stability,  robustness,  and  stabilization  for  a  class  of  nonlinear  systems,  originally 
studied  by  Feinberg,  Horn,  and  Jackson  in  the  1970s  and  80s,  which  arises  in  the  analysis  of 
biochemical  networks.  This  paper  extended  their  results,  to  obtain  global  stability  conclusions 
that  apply  to  the  kinetic  proofreading  model  for  T-cell  receptor  signal  transduction,  as  well  as 
provided  bounds  on  robustness  under  unstructured  perturbations  and  feedback  stabilization. 
The  class  of  nonlinear  systems  that  arises  in  this  fashion  is  very  different  from  others  typically 
considered  in  control  theory,  and  they  are  endowed  with  strong  robustness  properties. 

Furthermore,  when  outputs  are  explicitly  considered,  problems  of  filtering  can  be  posed. 
In  [s54]  and  [s71],  we  gave  necessary  and  sufficient  condition  for  detectability  and  an  explicit 
construction  of  globally  convergent  observers.  An  alternative  observer,  which  has  enhanced 
robustness  (expressed  through  an  ISS  property)  to  additive  disturbances,  was  studied  in  [s69]. 

Internal-Model  Principle 

One  strength  of  control  theory  is  that  it  tells  us  that  certain  structures  must  be  present  in 
systems,  in  order  for  regulation  objectives  to  be  met.  Such  insight  can  help  guide  experimental 
research,  as  it  suggests  trying  to  find  the  corresponding  subsystems,  and  may  thus  help  in 
model  validation.  Conversely,  the  absence  of  a  critical  subsystem  might  be  an  indication  that 
the  biological  entity  being  studied  does  not  regulate  its  behavior  in  some  hypothesized  sense. 

In  particular,  and  motivated  by  the  thought-provoking  paper  by  Doyle’s  group,  we  became 
interested  in  understanding  the  role  of  internal  models  in  biological  adaptation.  Recall  that  the 
classical  internal  model  principle  (IMP  due  to  Francis  and  Wonham)  states  that  if  controller 
regulates  a  system  against  external  disturbances  in  some  family,  and  if  this  regulation  is  struc¬ 
turally  stable  in  a  precise  mathematical  sense,  then  the  controller  must  necessarily  contain  a 
subsystem  which  can  itself  generate  all  such  disturbances  (and  which  is  driven  by  a  suitable 
“error”  signal).  A  potential  drawback,  when  attempting  to  use  this  theorem  in  biological  appli¬ 
cations,  is  that  the  structural  stability  criterion  may  be  impossible  to  check:  it  would  imply  for 
instance  regulation  even  in  the  presence  of  direct  connections  from  inputs,  such  as  cell  receptors, 
to  outputs,  such  as  flagellar  motors.  Thus,  once  again,  one  has  a  problem  which  “sounds”  just 
like  a  standard  problem  in  control  theory,  yet  which  mathematically  differs  in  a  subtle  fashion. 

In  [s41],  we  gave  an  internal  model  theorem  which  did  not  require  the  assumption  of  struc¬ 
tural  stability  (nor,  for  that  matter,  an  a  priori  requirement  for  the  system  to  be  partitioned 
into  separate  plant  and  controller  components).  Instead,  a  “signal  detection”  capability  is  im¬ 
posed.  These  weaker  assumptions  make  the  result  better  applicable  to  cellular  phenomena  such 
as  the  adaptation  of  Exoli  chemotactic  tumbling  rate  to  constant  concentrations.  The  “signal 
detection”  property  makes  a  lot  of  sense  in  biological  applications  (a  signal  must  be  detected 
with  very  high  gain,  and  an  action  commanded,  but  the  command  may  be  metabolically  too 
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expensive  to  be  kept  “on”  for  very  long)  but  not  necessarily  in  all  engineering  applications  (we 
would  not  want,  for  instance,  the  passengers  in  an  automobile  to  hit  the  car  roof  as  hard  as 
possible  before  an  active  suspension  system  takes  over  and  regulates  against  road  bumps). 

Adaptive  Control  of  Bifurcation  Parameters 

Some  biological  systems  are  believed  to  operate  at  a  critical  point  between  stability  and 
instability,  which  brings  up  the  issue  of  how  bifurcation  parameters  may  be  automatically 
tuned  to  this  critical  value.  The  papers  [s29]  and  [s30]  were  motivated  by  two  such  instances 
from  the  literature:  neural  integration  in  the  nervous  system,  and  hair  cell  oscillations  in  the 
auditory  system.  In  both  examples,  the.  question  arises  as  to  how  the  required  fine-tuning  may 
be  achieved  and  maintained  in  a  robust  and  reliable  way.  As  with  the  other  questions  illustrated 
here,  this  led  to  a  new  type  of  problem  in  control  theory,  related  but  in  fact  different  from  other 
work  on  “bifurcation  control”  in  the  literature.  We  formulated  this  question  in  the  language  of 
adaptive  control,  and  presented  solutions  in  some  simple  instances. 

1.3  Other  Nonlinear  Control  Research 

We  summarize  here  other  recent  research  accomplishments  by  the  PI  in  the  general  area  of 
nonlinear  systems. 

Control  of  Mechanical  Systems 

In  [s72]  and  [s46],  we  addressed  questions  of  time-optimal  control  for  mechanical  systems 
with  possible  dissipation  terms.  The  Lie  algebras  associated  with  these  systems  enjoy  certain 
special  properties.  These  properties  were  explored  and  are  used  in  conjunction  with  the  Pon- 
tryagin  TnaYimnm  principle  to  determine  the  structure  of  singular  extremals  and,  in  particular, 
time-optimal  trajectories.  The  theory  was  illustrated  with  an  application  to  a  time-optimal 
problem  for  a  class  of  imderwater  vehicles. 
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2  Selected  Topics 

Given  the  range  of  subjects  treated,  and  the  different  methodologies  employed  in  each  case,  we 
can  only  briefly  cover  a  few  selected  subtopics.  The  published  papers,  and  the  Pi’s  website 

http :  //www .  math .  rutgers .  edu/ -sontag 

provide  technical  details.  For  the  topics  treated,  we  do  not  repeat  here  most  of  the  introductory 
material  and  references  already  discussed  in  the  previous  section. 

2.1  Some  Details:  ISS  and  Related  Notions 

We  describe  first  topics  related  to  input  to  state  stability. 
loss  and  iISS  Characterizations 

Detectability  is  one  of  the  central  notions  of  control  theory.  It  plays  a  major  role  in  static 
state-feedback  analysis  (Lasalle’s  invariance  principle)  as  well  as  in  stabilization  by  means  of 
dynamic  output  feedback  and  in  observer  design.  Several  alternative  definitions  are  possible 
when  trying  to  precisely  define  detectability  in  the  context  of  nonlinear  control.  According 
to  the  specific  problem  under  consideration,  different  variants  are  useful  in  capturing  useful 
features  of  its  linear  counterpart.  One  version  which  has  proved  to  be  especially  powerful  for 
systems  subject  to  exogenous  disturbances,  is  to  define  this  notion  in  an  ISS  sense,  leading  to 
what  we  have  called  “input-output-to-state  stability”  (lOSS).  Such  a  notion  not  only  allows 
one  to  extend  Lasalle-type  stability  results  to  the  case  of  non-autonomous  systems,  but  it  also 
provides  a  machinery,  fuUy  compatible  with  the  formalism  of  input-to-state  stability,  that  helps 
one  understand  issues  such  as  minimum-phase  behavior  or  certainty  equivalence. 

Although  general  nonlinear  systems  may  often  exhibit  an  overwhelming  variety  of  behaviors, 
it  tirnis  out  that  many  other  “reasonable”  formulations  of  the  detectability  property  end  up 
being  equivalent  to  lOSS.  Recently,  we  proved  in  work  with  Krichman  and  Wang  in  [s64] 
that  loss  is  equivalent  to  a  dissipation  property.  In  the  even  more  recent  work  [sl6]  with 
Angeli,  Ingalls,  and  Wang,  we  were  able  to  characterize  lOSS  as  the  conjunction  of  several 
important  and  natural  weaker  notions,  in  essence  generalizing  the  fact  (for  differential  equations 
with  no  controls  nor  outputs)  that  global  asymptotic  stability  can  be  characterized  by  the 
combination  of  (neutral)  stability  and  attractivity.  For  systems  with  inputs,  such  generalizations 
are  nontrivial  because,  in  contrast  to  ordinary  (finite-dimensional)  differential  equations,  there 
is  no  compactness  to  appeal  to  (even  with  respect  to  weak  topologies  on  inputs,  since  we 
do  not  have  any  convexity  properties).  As  an  application  of  the  results,  we  also  obtained 
reformulations  of  the  notion  of  integral  input  to  state  stability  (iISS).  Proving  these  results 
forced  us,  in  turn,  to  establish  facts  of  independent  interest  in  differential  inclusions,  such  as  a 
generalization  of  the  Filippov- Wazewski  Relaxation  Theorem  to  infinite  time  intervals,  in  [s35] 
(see  also  [s56]  for  the  corresponding  theorem  in  Banach  spaces)  and  a  theorem  showing  that, 
for  locally  Lipschitz  difierential  inclusions,  uniform  and  non-uniform  global  asymptotic  stability 
are  equivalent,  cf.  [s4].  Let  us  give  an  outline  of  [sl6]  next. 

We  consider  systems  of  the  following  general  form: 


i(i)  =  f{x{t),u{t)),  y{t)  =  h[x{t)), 


(1) 
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where,  for  each  f  >  0,  x{t)  G  u{t)  G  It,  a  subset  of  E"^  which  here  we  take  for  simplicity  to 
be  all  of  E”'.  We  assume  that  the  maps  /  :  E^  x  E”'  ^  E^  and  /i :  E""  ^  E^  are  locally  Lipschitz 
continuous,  with  /(0,0)  =  0  and  h{0)  =  0.  The  symbol  H  denotes  the  usual  Euclidean  norms. 
By  an  input  we  mean  a  measurable  and  locally  essentially  bounded  function  u  :  X  — >  It,  where, 
X  is  a  subinterval  of  E  which  contains  the  origin.  Given  any  input  u  and  any  ^  G  E’^,  the  unique 
maximal  solution  of  the  initial  value  problem  x  =  /(x,tx),  x(0)  =  ^  (defined  on  some  maximal 
open  subinterval  of  X)  is  denoted  by  x(*,  u).  When  X  =  E>o,  this  maximal  subinterval  has  the 

form  [0,  The  system  is  said  to  be  forward  complete  if  for  every  initial  state  ^  and  for  every 
input  u  defined  on  E>o,  T^,u  =  The  corresponding  output  is  denoted  by  that 

is,  =  h{x{t,^^u))  on  the  domain  of  definition  of  the  solution.  The  Loo-norm  (possibly 

infinite)  of  a  function  v  defined  on  X  is  denoted  by  ||u||,  i.e.,  ||u||  =  (ess)sup{|u(t)|  G  X}.  In 
particular,  for  a  maximal  trajectory  x(-,^,w)  and  the  corresponding  output  function  y(*,^,w) 
of  (1)  defined  on  [0,T^,u),  ||u||,  ||xt|  and  ||?/j|  denote  the  Loo  norm  of  w(-),  and  y(*,C,  w) 

respectively  on  (We  make  a  slight  abuse  of  notation  and  use  sup  and  limsup  to 

mean  the  essential  supremum  where  appropriate.)  For  a  function  v  defined  on  an  interval  X, 
if  Xi  C  X,  we  use  to  denote  the  restriction  of  i;  to  Xi,  i.e.,  ^^^(t)  =  v{t)  if  t  G  Xi,  and 
(t)  =  0  otherwise.  We  use  standard  terminology  on  comparison  functions:  Af  is  the  class 
of  continuous,  increasing  functions  from  [0,  oo)  to  [0,  oo);  /C  is  the  set  of  Af  functions  7  that 
are  strictly  increasing  and  satisfy  7(0)  —  0;  ICoo  is  the  set  of  /C  functions  that  are  unbounded; 
£  is  the  set  of  functions  [0,  +00)  — ^  [0,  +00)  which  are  continuous,  decreasing,  and  converge 
to  0  as  their  argument  tends  to  +00;  ICC  is  the  class  of  functions  [0, 00)^  [O5O0)  which 

are  class  K  on  the  first  argument  and  class  £  on  the  second  one.  A  positive  definite  function 
7  :  [0, 00)  — >  [0, 00)  is  one  such  that  7(0)  =  ^0  and  7(5)  >  0  for  all  s  >  0.  The  following 
notions  were  introduced  in  our  previous  work:  The  system  (1)  satisfies  the  unboundedness 
observability  (UO)  property  if,  for  each  state  ^  and  control  u  such  that  T^^u  <  00,  it  holds  that 
lim sup^_,.7’^  ^  \y{t,^,u)\  =  +00,  that  is,  for  each  state  ^  and  control  n,  T^^u  <  00  =>  ||2/||  =  +00. 
The  system  (1)  is  input-output-to-state  stable  (lOSS)  if  there  exist  some  p  G  /C£,  7,1  G  /C  and 
jy  £  1C  such  that 

<  /3(kl  )<)  +  7u(IHI)  +  7y  (|l2/[0,t]||)  (2) 

for  all  t  e  [0,Tf,„),  all  ^  6  R"  and  aU  u{-).  Clearly,  the  lOSS  property  implies  the  UO  property. 
The  following  local  version  is  also  important:  The  system  (1)  is  locally  lOSS  (local  lOSS)  if 
there  exist  ^  >  0  and  functions  f3  €  ICC,  7u,  7y  €  ^  so  that  for  any  ^  €  R”,  any  u(-), 

max{|^|,|M|,|M|}  <<5  =>  !x(t,^,«)|  </3(|e|,0  +  7«(IMI)  +  7y(l|y[o.t]ll)Vt  e  (3) 

We  also  say  that  the  system  (1)  satisfies  the  input-output  limit  property  (lO-LIM)  if  for 
some  7„,  7y  e  K, 

mi  |x(<,^,u)|  <max{7u(||u|l),7y(||y||)} 
tG[0,T^,u) 

for  all  ^  G  E”  and  all  inputs  u(-),  that  it  satisfies  the  input-output  asymptotic  gain  property 
(lO-AG)  if  solutions  are  ultimately  bounded  by  some  nonlinear  gain  function  of  ||u||  and  ||2/||, 
that  is,  for  some  7^,  7y  G  /C, 

limsup  I  <  max{7„(||u|l),7y(||y||)} 
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for  all  ^  €  E"  and  all  inputs  u(-);  that  it  satisfies  the  input- output-to-state  boundedness  property 
(lO-BND)  if  for  some  co,  <Ty  e  M,  it  holds  that 

|x(t,C,u)|  <  max{(To(|^I),<T„(||u||),CTy(||y[o,t]||)}, 

for  all  ^  e  K"  and  all  inputs  «(•),  and  all  t  £  [0,  and  finally  that  it  satisfies  the  input-output 
global  stability  property  (lO-GS)  if  the  functions  tro,  o'u,  CFy  above  can  be  taken  to  be  of  class  K,. 
(It  is  not  hard  to  see  that  each  of  the  10- AG,  the  lO-GS,  and  the  lO-BND  properties  implies 
the  UO  condition.) 

Thinking  of  these  detectability  properties  as  “stability  modulo  inputs  and  outputs”,  we 
can  identify  lOSS  with  asymptotic  stability,  lO-GS  with  (neutral)  stability,  and  lO-AG  with 
attractivity.  In  this  context,  it  seems  perfectly  natural  that  lOSS  should  be  equivalent  to  the 
combination  of  lO-GS  and  10- AG,  and  indeed  that  is  one  of  the  dwompositions  which  appears 
in  our  main  result.  Related  results  follow  by  considering  other  “basic”  stability-hke  notions, 
such  as  lO-LIM.  Among  many  others,  the  paper  [sl6]  proves  equivalences  among  the  following 
statements: 


•  (loss) 

•  (lO-AG)  &  (lO-GS) 

•  (lO-AG)  &  (local  loss) 

•  (lO-LIM)  &  (lO-GS) 

•  (lO-LIM)  &  (local  loss) 


We  then  went  on  to  derive  asymptotic  characterizations  of  the  integral  input-to-state  stability 
property.  Recall  that  we  defined  a  system  to  be  integral  input-to-state  stable  (iISS)  if  there 
exist  functions  p  £  K-C,  c  £  K,  and  'y  £  K,,  such  that,  for  all  ^  £  E”  and  all  u,  the  solution 
x{t,  u)  is  defined  for  alH  >  0,  and 

for  all  t  >  0.  In  order  to  obtain  an  interesting  separation  theorem,  we  introduced  in  [sl6]  the 
following  properties.  We  say  that  a  system  (1)  satisfies  the  Bounded  Energy  Weakly  Converging 
State  (BEWCS)  property  if  for  some  cr  of  class  Kooi  h  holds  that 

r+oo 

/  <T(|u(s)|)ds  <  -foo  Uminf  |a;(t,^,u)|  =  0. 

Jo  t->+oo 

(To  be  more  precise,  this  means  that  for  any  ^  and  any  u  for  which  cr(|u(s)|)ds  <  oo,  it 
holds  that  =  oo,  and  Iiminfi_+oo  |x(t,^,u)|  =  0.)  We  say  that  it  satisfies  the  Bounded 
Energy  Frequently  Bounded  State  (BEFBS)  property  if  for  some  o  of  class  Kcx,,  h  holds  that 
/•+00 

/  o-(lu(s)|)ds  < -1-00  =»  liminf  |x(<,^,u)|  < -1-00. 

Jo  t->+oo 

(To  be  more  precise,  for  any  ^  and  any  u  for  which  <t(|u(s)|)  ds  <  oo,  it  holds  that  T^^u  =  oo, 
and  Uminft-4+oo  |a!(t,^,ti)l  <  oo.)  (BEFBS  and  BEWCS  esich  implies  forward  completeness.) 
Finally,  we  say  that  a  system  as  in  (1)  is  zero-GAS  if  the  corresponding  zero-input  system 
X  =  f{x,  0)  is  globally  asymptotically  stable;  that  it  is  zero-LS  if  the  zero-input  system  is  locally 
(neutrally)  stable;  and  that  it  is  zero-LAS  if  the  zero-input  system  is  locally  asymptotically 
stable.  Then,  we  proved  the  equivalence  among  the  following  statements: 
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•  iISS 

•  BEWCS  and  zero-LS 

•  BEFBS  and  zero-GAS 

These  results  complete  the  theory  of  lOSS  and  iISS  in  very  elegant  ways,  and  make  the  con¬ 
cepts  much  easier  to  verify  in  applications.  We  will  next  introduce  another  important  systems 
property,  measurement  to  error  stability  (MES). 

MES 

When  discussing  systems  with  outputs,  the  output  signal  typically  plays  one  of  two  roles. 
One  is  that  in  which  the  outputs  are  considered  as  measurements.  Here,  one  supposes  that 
knowledge  of  the  whole  state  is  not  available,  but  rather  that  only  partial  knowledge  of  the 
state  can  be  used,  and  the  output  is  meant  to  provide  information  about  the  state.  This  leads 
to  our  detectability  notion  of  output  to  state  stability  (OSS).  A  second  role  for  outputs  occurs 
when  the  goal  of  the  control  design  is  not  to  regulate  the  behavior  of  the  entire  state,  but 
rather  only  to  regulate  the  output  signal.  The  theory  of  output  regulation  addresses  precisely 
this  situation,  that  of  keeping  an  output  small.  This  leads,  within  the  ISS  framework,  to  the 
notion  of  stability  of  the  output  signal  described  by  input  to  output  stability  (lOS).  (In  the 
case  of  systems  with  no  inputs,  the  problem  of  stability  of  a  subset  of  the  state  variables  is  the 
special  case  of  stability  of  an  output  signal  which  is  a  projection,  and  it  has  been  addressed  in 
the  ordinary  differential  equations  literature  under  the  name  “partial  stability”  by  Vorotnikov 
and  others.) 

Consider  now  the  case  in  which  both  the  above  situations  occur.  That  is,  there  are  two 
output  signals,  one  which  is  measured,  and  the  other  which  must  be  regulated.  A  special 
case  of  this  situation  has  been  addressed  in  the  output  regulation  theory,  under  the  name 
“error  feedback”.  This  theory  formulates  the  question  of  regulating  an  output  of  the  system 
(the  error)  with  knowledge  of  that  output  only.  The  more  general  case  is  when  there  axe  two 
distinct  channels  playing  these  two  roles.  In  the  paper  [s58],  we  generalized  the  notion  of  OSS 
to  this  situation  by  introducing  the  concept  of  measurement  to  error  stability  (MES),  which 
can  be  viewed  as  a  notion  of  partial  detectability  through  the  measurement  channel.  We  gave 
a  partial  Lyapunov  characterization  of  the  MES  property,  accomplished  by  first  comparing  the 
MES  property  to  a  notion  of  output  stability  relative  to  a  set.  This  notion,  which  we  called 
stability  in  three  measures  (SIT)  was  characterized  by  the  existence  of  a  lower  semicontinuous 
Lyapimov  function.  It  was  shown  that  the  SIT  property  implies  MES,  and  that  the  converse 
holds  imder  an  additional  boundedness  assumption.  The  analysis  was  carried  out  on  systems 
described  by  differential  inclusions  -  implicitly  incorporating  a  disturbance  input  with  compact 
value-set.  Let  us  now  give  some  details.  We  consider  the  differential  inclusion 

x{t)  e  F{x{t))  (4) 


with  two  output  maps 

y{t)  =  h{x{t)),  w{t)  =  g{x{t)), 

and  a  map  a;  :  K”  — >  R>o-  We  take  the  state  x  €  E".  We  assume:  the  set-valued  map 
F  from  R”  to  subsets  of  R”  is  locally  Lipschitz  with  nonempty  compact  values,  the  output 
maps  ft  :  R”  — >  R^"  and  5  :  R”  -+  R^*"  are  locally  Lipschitz,  the  map  w  (to  be  used  as  a 
measurement  of  the  magnitude  of  the  state  vector)  is  assumed  continuous  and  proper,  and  the 
differential  inclusion  (4)  is  forward  complete.  We  will  denote  l  :=  a;(  ).  (The  use  of  11^^ 
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allows  a  framework  which  includes  the  Euclidean  norm,  distance  to  a  compact  set,  and  more 
general  measures  of  the  magnitude  of  the  state.)  For  each  C  C  we  let  S{C)  denote  the  set  of 
TnaYimal  solutions  of  (4)  satisfying  a:(0)  €  C  equipped  with  the  topology  of  uniform  convergence 
on  compact  intervals.  If  C7  is  a  singleton  {^},  we  use  the  shorthand  S{^).  We  set  S  :=  5(R”), 
the  set  of  all  m8odmal  solutions.  Given  a  trajectory  x(-)  G  5(^)  for  some  ^  €  R”,  we  denote 
y(t)  =  h[x{t)  and  w{t)  =  g{x{t)),  for  all  t  >  0. 

Our  main  concept  is  the  following  one.  We  say  that  the  system  (4)  is  measurement  to  error 
stable  (MES)  if  there  exist  0  G  ICC  and  7  G  /C  so  that 

|y(t)|  <  max{^(|x(0)|^,t),7(lkll[o,t])} 

for  each  x{-)  G  5,  and  alH  >  0.  In  the  investigation  of  the  MES  property,  the  following  notion 
of  relative  stability  of  the  error  will  be  useful.  This  is  a  notion  of  output  stability  which  is 
applicable  to  systems  with  a  single  output  y.  Given  a  closed  subset  D  of  the  state  space  R",  we 
say  that  the  system  (4)  is  relatively  error  stable  (RES)  with  respect  to  D  if  there  exists  ^  G  KC 
so  that  for  any  solution  x(  )  G  S',  if  there  exists  <1  >  0  so  that,  if  x{t)  ^  for  all  t  G  [0,ti], 
then 

\y{t)\<P{\x{0)l,t)  yte[0,hl  (5) 

A  special  case  of  this  property  occurs  for  a  system  with  two  outputs  when  the  set  D  is  defined  by 
an  inequality  involving  the  two  output  maps,  as  follows.  Let  p  £  fC.  We  say  that  the  system  (4) 
satisfies  the  stability  in  three  measures  (SIT)  property  (with  gain  p)  if  there  exists  p  e  ICC  so 
that  for  any  solution  x(*)  £  5,  if  there  exists  >  0  so  that,  if  |2/(t)|  >  p(|u;(t)|)  for  all  t  £  [0,  ti], 
then  (5)  holds.  It  is  immediate  that  SIT  is  equivalent  to  relative  error  stability  with  respect  to 
the  set  D  :=  {^  E  E”  :  \h{^)\  <  p(b(^)|)}.  Finally,  we  also  introduced  in  [s58]  the  following 
relative  stability  property.  The  system  (4)  satisfies  the  relative  measurement  to  error  bounded 
property  (RMEB)  if  there  exist  K  functions  pi,  (Ti,  and  cf2  so  that  for  any  solution  x(-)  E  5,  if 
there  exists  ti  >  0  so  that,  if  |2/(t)|  >  Pi(lti^(t)l)  for  all  t  £  [0,ti],  then  for  all  t  E  [0,ti], 

ly(t)|  <  max{(7i(|^(x(0))|),<T2(|tt/;||jo,t])}. 

We  provided  dissipation  characterizations  as  follows.  Given  an  open  set  E  C  a  lower 
semicontinuous  function  V  :  E>o  is  an  RES-Lyapunov  function  for  system  (4)  on  E  if 

there  exist  ai,  02  E  ICoo  so  that  for  all  ^  E  J5,  0'i(|/i(^)|)  <V{^)<  QJ2(|^L)i  there  exists 
<^3  :  E>o  E>o  continuous  positive  definite  so  that  for  each  ^  E  J5,  C  •  t;  <  -asiViO)  for  all 
C  E  doV{^)  and  all  v  £  F{^)^  where  we  axe  denoting  by  9dV'(^)  the  viscosity  subdiflferential 
of  V  at  i.e.  the  (possibly  empty)  set  of  viscosity  subgradients  of  V  at  We  say  that  V  is 
an  exponential  decay  RES-Lyapunov  function  for  system  (4)  on  E  if  this  property  holds  with 
Q;3(r)  =  r.  We  also  specialize  the  above  definitions  for  the  notion  of  stability  in  three  measures. 
Let  p  £  K.  A  lower  semicontinuous  function  V  :  ^  E>o  is  an  SIT-Lyapunov  function  for 

system  (4)  with  gain  p  if  there  exist  oi,  02  E  JCoo  so  that  for  each  ^  so  that  \h{^)\  >  p(|y(C)l),  it 
follows  that  ai(l^(OI)  <  ^(0  <  Q;2(|^L)»  and  there  exists  03  :  E>o  ^  E>o  continuous  positive 
definite  so  that  for  each  ^  so  that  \h{^)\  >  /?(|^(0I)»  C  *  ^  "“^3(I^(^))  for  all  C  £  dDV{^)  and 

all  v  £  F{^).  Finally,  V  is  an  exponential  decay  SIT-Lyapunov  function  for  system  (4)  with 
gain  p  if  in  addition  this  holds  with  03  (r)  =  r. 

We  then  obtained  the  following  theorem.  For  any  system  (4)  and  closed  set  D  C  E”,  and 
denoting  E  =  R^\D,  the  following  are  equivalent: 
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•  The  system  is  relatively  error  stable  with  respect  to  D 

•  The  system  admits  an  RES-Lyapunov  function  on  E 

•  The  system  admits  an  exponential  decay  RES-Lyapunov  function  on  E 

As  a  corollary  (set  D  =  G  R"  :  |h(OI  ^  have,  for  any  p  E  K,,  that  the 

following  are  equivalent; 

•  The  system  satisfies  the  SIT  property  with  gain  p 

•  The  system  admits  an  SIT-Lyapunov  function  with  gain  p 

•  The  system  admits  an  exponential  decay  SIT-Lyapunov  function  with  gain  p 

This  provides  a  characterization  of  the  SIT  property,  which  is  related  to  MES  as  follows:  MES 
implies  SIT,  and  SIT  and  RMEB  together  imply  MES.  In  conclusion,  if  (4)  satisfies  MES,  then 
it  admits  an  exponential  decay  SIT-Lyapunov  function.  Conversely,  if  it  satisfies  the  RMEB 
property  and  it  admits  an  SIT-Lyapunov  function,  then  it  satisfies  MES. 

ISS  with  Respect  to  Derivatives 

Let  us  now  turn  to  the  derivative-ISS  property  that  we  studied  in  [s43].  In  the  ISS  literature, 
inputs  (thought  of  as  “disturbances”)  are  arbitrary  (locally  essentially  bounded  and  Lebesgue- 
measurable)  functions.  Such  an  extremely  rich  set  of  possible  input  perturbations  is  well  suited 
to  model  noise,  as  well  as  constant  or  periodic  signals,  slow  parameter  drift,  and  so  on.  If, 
on  the  one  hand,  this  makes  the  notion  of  ISS  extremely  powerful,  on  the  other  hand  it  is 
known  that  ISS  might  sometimes  be  too  stringent  a  requirement.  In  the  output  regulation 
literatme,  instead,  the  focus  is  often  on  “deterministic”  disturbances,  i.e.,  signals  that  can 
be  generated  by  a  finite  dimensional  nonlinear  systems,  (usually  smooth).  This  is  a  class  of 
persistent  disturbances  for  which,  roughly  speaking,  the  following  is  true: 

||d||oo  small  =4>  ||ti||oo  and  derivatives  of  arbitrary  order  are  also  small. 

Under  similar  circumstances,  for  instance  when  cascading  asymptotically  stable  systenas,  re¬ 
garding  the  “forcing”  system’s  state  as  a  disturbance  typically  yields 

lim  sup  \d{t)  I  =  0  ==;>■  lim  sup  ld(t)  |  =  0. 

^4-00  t— +H-0O 

Nevertheless,  the  classical  definition  of  input-to-state  stability  completely  disregards  such  ad¬ 
ditional  information.  Tiacking  of  output  references  is  yet  another  area  where  “derivative” 
knowledge  is  usually  disregarded  (the  analysis  is  often  performed  only  taking  into  accoimt  con¬ 
stant  set-points),  whereas  such  information  could  be  exploited  to  get  tighter  estimates  for  the 
steady-state  tracking  error  due  to  time- varying,  smooth  reference  signals.  An  analogous  situ¬ 
ation  also  arises  when  parameter  variations  are  taken  into  account  (in  adaptive  control),  and 
we  expect  the  system  to  have  suitably  stable  behavior  for  slow  parameter  drifts.  The  study  of 
systems  with  slowly  varying  parameters  has  long  been  an  focus  of  research.  The  analysis  of 
such  a  system  is  usually  carried  out  by  first  considering  the  systems  corresponding  to  “frozen” 
parameters.  If  for  all  frozen  parameters,  the  corresponding  frozen  systems  uniformly  posses 
certain  stability  properties,  then  it  is  reasonable  to  expect  that  the  system  with  slowly  varying 
parameters  will  posses  the  same  property.  A  more  general  question  is  how  the  magnitudes  of 
the  time  derivatives  of  the  time  varying  parameters  affect  the  behavior  of  the  systems.  In  [s43], 
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we  offered  an  ISS-like  stability  notion  which  takes  into  account  robustness  with  respect  to  dis¬ 
turbances  and  their  time  derivatives.  The  new  notion  of  “D'^ISS"  is  defined  through  an  estimate 
which  involves  the  magnitudes  of  the  inputs  and  their  derivatives  up  to  the  fc-th  order.  We  also 
proposed  several  properties  related  to  the  D^'ISS  notion.  All  these  properties  serve  to  formalize 
the  idea  of  “stable”  dependence  upon  the  inputs  and  their  time  derivatives.  They  differ  in  the 
formulation  of  the  decay  estimates  which  make  precise  how  the  magnitudes  of  derivatives  affect 
the  system.  Finally,  we  illustrated  by  examples  how  these  properties  differ  from  each  other  and 
from  ISS  itself.  Let  us  give  some  details  now. 

We  denote  by  for  any  integer  A:  >  1  and  any  interval  J,  the  Sobolev  space 

consisting  of  all  functions  u  :  J  ->  E*”  for  which  the  {k  -  l)st  derivative  exists  and 

is  locally  Lipschitz.  For  A:  =  0,  we  define  as  the  set  of  locally  essentially  bounded 

u  :  J  ^  R”*.  When  J  =  [0, -boo),  we  omit  J  and  write  simply  We  say  that  the 

system  (1)  is  kth  derivative  input-to-state  stable  (D^'ISS)  if  there  exist  some  /C£-function  P, 
and  AC-functions  70, 71,  •  ,Jk  such  that,  for  every  input  u  S  the  following  holds: 

\x{t,^,u)\  <  /?(|^|  ,<)  -f  7o(1|m||)  +  7l(ll«ll)  +  •  •  •  +  7fc  (l|w^''^ll) 

for  all  t  >  0.  We  say  simply  that  the  system  is  DISS  when  it  is  £>^ISS  and,  of  course,  ISS  is 
the  same  as  D'^ISS  for  A:  =  0.  A  dissipation  characterization  is  as  follows.  Let  A:  >  1.  The 
system  (1)  is  D'^ISS  if  and  only  if  there  exists  a  smooth  function  F  :  E”  x  E*’'”  — >  E>o  such 
that 


•  there  exist  some  a^aeKoo  such  that  for  all  (x,  €  E”  x  E*''",  it  holds  that 

•  there  exist  some  a  €  ICoaP  €  A^oo  such  that  for  all  x  €  E"  and  all  with 

=  (mo,  Ml,  •  •  • ,  Mfc),  it  iioifis  that 

^(•)/(^,Mo)  +  ^()mi  +  |^(•)M2+  •••^(■)Mfc<-«(n-))  +  P(lM'*'^l) 

(where  “(•)”  means  (x,/xl*^“^i)). 

This  provides  a  useful  test  for  the  property,  and  is  helpful  in  proving  a  separation  property 
analogous  to  those  proved  for  lOSS  and  iISS  and  cited  earlier.  Clearly,  if  a  system  is  ZA^'ISS, 
then  it  is  forward  complete  (for  u  e  and  for  some  70, 71, . . . ,  7*;  G  AC  it  holds  that 

limsup|x(t,^,u)|  <  7o(||w||)  +7i(ll“ll)  H - f  7fc(lk^*’^ll)- 

t— >CX) 

If  this  holds,  for  some  70, ...  ,7*:  €  AC,  for  all  ^  G  E”  and  all  u  G  we  say  that  a  system 

satisfies  the  k-asymptotic  gain  (A:- AG)  property.  The  less  obvious  and  more  interesting  con¬ 
verse  was  also  shown  to  be  true,  namely,  for  a  forward  complete  system  (1),  the  following  are 
equivalent: 

.  It  is  jD*'ISS 

•  It  satisfies  the  A:- AG  property  and  the  zero-input  system  x  =  /(x,  0)  is  (neutrally)  stable 
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Many  other  results  are  proved  in  [s43],  regarding  stability  properties  of  cascades  as  well  as 
further  characterizations.  We  also  show  there  that  the  DISS  property  is  strictly  weaker  than 
plain  ISS,  which  is  of  course  not  surprising.  (Although,  interestingly,  the  two  properties  are 
equivalent  for  systems  of  dimension  one.)  Also,  obviously  DISS  implies  an  ISS  property  for 
constant  inputs,  but  it  is  perhaps  less  obvious  that  the  converse  does  not  hold  (ISS  with  respect 
to  constant  inputs  does  not  imply  DISS). 

Cascades 

We  now  turn  to  some  recent  work  on  cascaded  subsystems,  with  Arcak  and  Angeli,  reported 
in  [s55].  Constructive  design  methods  such  as  backstepping  and  forwarding,  which  are  based  on 
recursive  applications  of  cascade  designs,  together  with  the  discovery  of  structural  obstacles  to 
stabilization  such  as  the  “peaking  phenomenon”  by  Sussmann  and  Kokotovid,  have  paved  the 
road  to  major  advances  in  nonlinear  control.  One  of  the  main  motivations  for  the  stabilization 
of  cascades  came  from  the  linear-nonlinear  cascade 

X  =  f{x,z)  (6) 

z  =  Az  +  Bu  (7) 

resulting  from  input-output  linearization.  Because  global  asymptotic  stability  (GAS)  of  the 
x-subsystem  x  =  /(x,  0)  is  not  sufficient  to  achieve  GAS  of  the  whole  cascade  with  2-feedback 
u  =  Kz,  alternative  designs  which  employ  x-feedback  were  developed,  such  as  the  “feedback 
passivation”  design  of  Kokotovid  and  Sussmann.  To  achieve  GAS  by  2-feedback,  several  authors 
studied  general  cascades  in  which  the  2-subsystem  is  nonlinear,  and  derived  conditions  for  the  x 
and  2-subsystems  that  ensure  stability  of  the  cascade.  Among  these  results,  a  particularly  useful 
one  is  that  if  the  x-subsystem  is  ISS  with  input  2,  and  the  2-subsystem  is  GAS,  then  the  cascade 
is  GAS.  This  result  has  been  widely  used  for  nonlinear  designs  based  on  the  normal  form,  in 
which  the  zero  dynamics  (6)  is  ISS.  Other  results  make  less  restrictive  assumptions  than  ISS 
for  the  x-subsystem,  but  restrict  the  2-subsystem  to  be  locally  exponentially  stable  (LES).  On 
the  other  hand,  since  the  iISS  property  is  less  restrictive  than  ISS  (because,  in  an  iISS  system, 
a  bounded  input  may  lead  to  unbounded  solutions  if  its  energy  norm  is  infinite),  [s55]  analyzed 
the  stability  of  nonlinear  cascades  in  which  the  x-subsystem  is  merely  iISS.  The  admissible  iISS 
gains  for  stability  were  diaracterized  from  the  speed  of  convergence  of  the  2-subsystem.  When 
the  convergence  is  fast,  the  iISS  gain  function  of  the  x-subsystem  is  allowed  to  be  “steep”  at 
zero.  The  paper  showed  that  this  trade-off  between  slower  convergence  and  steeper  iISS  geiin 
encompasses,  and  unifies,  several  results  in  the  literature.  In  particular,  if  the  x-subsystem  is 
ISS  then  the  slope  of  its  iISS  gain  function  is  very  gentle  at  zero,  and  tolerates  every  GAS 
2-subsystem  no  matter  how  slow  its  convergence  is.  On  the  other  hand,  if  the  convergence  is 
exponential,  that  is  if  the  2-subsystem  is  LES,  then  the  cascade  is  stable  for  a  large  class  of 
iISS  gains.  This  class  includes  all  iISS  x-subsystems  that  are  affine  in  the  input  z.  Thus,  for 
systems  like  (6)- (7),  where  a  control  law  can  be  designed  to  render  the  2-subsystem  GAS  and 
LES,  the  iISS  property  of  the  x-subsystem  ensures  GAS  of  the  cascade.  Several  results  in  the 
literature  are  special  cases  of  the  main  result,  including  those  that  restrict  the  x-subsystem  to 
be  ISS  and  those  that  restrict  the  2-subsystem  to  be  LES. 

The  paper  [s55]  also  gave  an  output-feedback  application  of  the  cascade  result,  in  the  style 
of  other  work  by  Arcak.  Due  to  the  absence  of  a  separation  principle,  it  is  necessary  to  design 
control  laws  that  guarantee  robustness  against  the  observer  error.  A  design  was  presented 
which  renders  the  system  iISS  with  respect  to  the  observer  error  and,  hence,  ensures  robustness 
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when  the  error  is  exponentially  decaying.  The  advantage  of  this  design  over  the  more  classical 
observer-based  backstepping  scheme  is  that  it  employs  “weak”  nonlinear  damping  terms  which 
grow  slower  than  previous  ones,  and  result  in  a  “softer”  control  law.  Far  more  work  is  stih 
required  in  this  area,  resulting  in  more  general  observer  designs  that  make  systems  iISS  with 
respect  to  observer  error;  see  the  latter  discussion  regarding  separation  principles,  as  well  as 
the  remarks  about  observers  for  chemical  networks  (which  rely  on  an  ISS  property,  as  cmrrently 
stated,  and  hence  could  potentially  be  greatly  improved). 

ISS  Stabilization  Under  Feedback 

A  different  direction  of  research  is  that  in  our  work  with  Malisoff  and  Rifford,  reported 
in  [sl4],  which  dealt  with  the  ISS-stabihzation  of  systems 

X  =  f{x)  +  G{x)u  (8) 

(where  /  and  G  are  locally  Lipschitz  vector  fields  on  R”,  /(O)  =  0,  and  the  control  u  is  valued 
in  K”*;  we  also  studied  possible  extensions  to  fully  nonlinear  systems).  The  main  theorems 
show  that,  if  the  system  (8)  is  globally  asymptotically  controllable  (GAC),  then  there  exists  a 
feedback  AT :  E"  ^  E”*  for  which 

X  =  fix)  +  Gix)K{x)  +  G{x)u  (9) 

is  ISS,  which,  when  written  as  x  =  /(x)-|-G(x)(Ar(x)-|-u),  can  be  interpreted  as  robustness  with 
respect  to  actuator  errors.  Recall  that  one  of  the  original  motivations  for  the  introduction  of  the 
notion  of  ISS  was  to  give  a  precise  formulation  of  the  fact  that  stabilizable  systems  may  be  made 
also  stable  with  respect  to  actuator  errors.  This  was  central  to  showing  the  existence  of  coprime 
factorizations  for  classes  of  systems  including  feedback  linearizable  systems.  Since  a  continuous 
stabilizing  feedback  K  fails  to  exist  in  general,  one  is  forced  to  consider  discontinuous  feedbacks 
AT,  so  solutions  of  (9)  are  understood  in  the  senses  of  sampling  and  of  Euler  solutions  (each 
type  of  solution  is  covered  by  a  different  theorem).  The  results  in  [sl4]  extended  our  original 
ones  (1989),  which  dealt  with  the  case  when  the  original  system  is  stabilizable  with  regular 
feedback  (interestingly,  however,  the  actual  definition  of  K  is  different,  in  the  new  paper,  even 
when  specialized  to  the  case  covered  by  the  older  work).  In  particular,  our  results  apply  to  the 
example  of  Brockett’s  nonholonomic  integrator. 

It  was  a  long-standing  problem  to  obtain  such  generalizations,  and  they  heavily  rely  upon  the 
new  techniques  involving  semiconcave  control  Lyapunov  functions.  Our  results  also  strength¬ 
ened  our  paper  1997  with  Clarke  et  al.,  which  constructed  feedbacks  for  GAC  systems  which 
render  the  closed  loop  systems  globally  asymptotically  stable.  They  apply  in  the  more  general 
situation  where  measurement  noise  may  occur.  In  particular,  our  feedback  K  has  the  additional 
feature  that  the  perturbed  system 

X  =  fix)  -1-  G(x)A:(x  -I-  e)  +  G(x)u  (10) 

is  also  ISS  when  the  observation  error  e  :  [0,  oo)  — >  E”  in  the  controller  is  sufficiently  small. 
In  this  context,  the  precise  value  of  e(t)  is  unknown  to  the  controller,  but  information  about 
upper  bounds  on  the  magnitude  of  e(t)  can  be  used  to  design  the  feedback.  We  showed  these 
theorems:  (1)  If  (8)  is  GAC,  then  there  exists  a  feedback  K  for  which  (10)  is  ISS  for  Euler 
solutions;  (2)  If  system  (8)  is  GAC,  then  there  exists  a  feedback  AT  sudi  that  (10)  is  ISS  for 
sampling  solutions.  The  first  theorem  characterizes  the  uniform  limits  of  sampling  solutions  of 
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(10);  from  a  computational  standpoint,  it  is  also  desirable  to  know  how  frequently  to  sample  in 
order  to  achieve  ISS  for  sampling  solutions.  This  information  is  provided  by  the  second  result. 

Let  us  provide  some  details  of  [sl4].  For  each  fc  G  N  and  r  >  0,  we  let  be  the  set  of 
measurable  u  :  [0,  oo)  — >  R*'  with  are  essentially  bounded,  ■=  {u  €  M'^  :  |«|oo  <  r})  where 
I  -  loo  is  the  essential  supremum,  and  rBk  •=  {a:  €  R*^  :  |x|  <  r}  for  each  k  E  N  and  r  >  0, 
whose  closure  is  rBk-  We  also  set  O  :=  {e  :  [0, oo)  — >  R"},  sup(e)  =  sup{|e(<)l  :  t  >  0}  for 
all  e  e  O,  and  Or,  ■=  {e  E  O  :  sup(e)  <  77}  for  each  77  >  0.  For  any  compact  set  .F  C  R"  and 
e  >  0,  we  define  the  compact  set  :=  {x  G  R”  :  inf{|x  —  p|  :  p  E  F}  <  e}.  Given  a  continuous 
function  h  :  R"  x  R"*  R”  :  (x,  u)  1-^  h{x,  u)  which  is  locally  Lipschitz  in  x  imiformly  on 
compact  subsets  of  R”  x  R*",  we  let  Xg,  u)  denote  the  trajectory  of  x  =  h{x,  u)  starting  at 
Xo  €  R"  for  each  u  E  M'^.  In  this  case,  4)h{-,Xo,u)  is  defined  on  some  maximal  interval  [0,t), 
with  t  >  0  depending  on  u  and  Xg-  A  system  x  =  h{x,u)  is  said  to  be  globally  asymptotically 
controllable  (GAC)  provided  there  exist  a  nondecreasing  function  a  :  [0,oo)  ->  [0,oo)  and  a 
function  (3  E  KC  satisfying  the  following:  For  each  Xg  E  R”,  there  exists  u  E  such  that 
\<t>h{t,Xg,u)\  <  P{\xg\,t)  for  all  t  >  0;  and  |u(t)|  <  <r(|<^/,(t,Xo,tt)|)  for  a.e.  f  >  0.  In  this 
case,  we  call  cr  the  GAC  modulus  of  x  =  h{x,u).  In  om:  main  results,  the  controllers  are 
discontinuous  feedbacks,  so  the  dynamics  will  be  discontinuous  in  the  state  variable.  Therefore, 
we  form  trajectories  through  sampling,  and  through  uniform  limits  of  sampling  trajectories, 
as  follows.  We  say  that  tt  =  {to,ti,t2,-..}  C  [0,oo)  is  a  partition  of  [0,oo)  provided  tg  =  0, 
ti  <  ti+i  for  alH  >  0,  and  ->  00  as  i  ->  +00.  The  set  of  all  partitions  of  [0,  oo)  is  denoted 
by  Par.  Let  F  :  R"  x  R'"  x  R"*  ^  R"  :  (x,p,u)  1-^  F{x,p,u)  be  a  continuous  function  which 
is  locally  Lipschitz  in  x  vmiformly  on  compact  subsets  of  R"  x  R”*  x  R"».  A  feedback  for  F  is 
defined  to  be  any  locally  bounded  function  if  :  R"  R"*  for  which  if(0)  =  0.  In  particular, 
we  allow  discontinuous  feedbacks.  The  arguments  x,  p,  and  u  in  F  axe  used  to  represent 
the  state,  feedback  value,  and  actuator  error,  respectively.  Given  a  feedback  if  :  R”  — >  R”*, 
TT  =  {to,ti,t2, . . .}  G  Par,  Xg  G  R",  e  G  O,  and  u  E  M^,  the  sampling  solution  for  the  initial 
value  problem  (IVP) 

x(t)  =  F(x(t),K(x(t) +  e(t)),u(t))  (11) 

X(0)  =  Xg  (12) 

is  the  continuous  function  defined  by  recursively  solving  x(t)  =  F(x(t),K(x(ti)  +  e{ti)),u{t)) 
from  the  initial  time  ti  up  to  time  Sf  =  sup{s  G  [ti,ti+i]  :  x(  )  is  defined  on  [<t,s)},  where 
x(0)  =  Xg.  In  this  case,  the  sampling  solution  of  (11)-(12)  is  defined  from  time  zero  up  to  time 
t  =  inf{st :  Si  <  tj+i}-  This  sampling  solution  will  be  denoted  by  1 1->  x-n{t\Xo,u,e)  to  exhibit 
its  dependence  on  tt  G  Par,  Xg  G  R”,  n  G  M,  and  e  G  O,  or  simply  by  x,r  when  the  dependence 
is  clear  from  the  context.  In  particular,  if  Si  =  ti+i  for  all  i,  then  i  =  +00  (as  the  infimum  of 
thie  empty  set),  so  in  that  case,  the  sampling  solution  t  x-,r{t-,Xg,u,e)  is  defined  on  [0, 00). 
We  also  define  the  upper  diameter  and  lower  diameter  of  a  partition  tt  =  {tg,ti,t2,  ■  ■  ■}  by 
d(7r)  =  sup(ti+i  -  ti)  and  d(7r)  =  inf(ti+i  -  ti)  (sup  and  inf  over  i  >  0)  respectively.  We  let 
Par(5)  :=  {tt  G  Par  :  3(7r)  <  5}  for  each  ^  >  0.  We  say  that  a  function  y  :  [0, 00)  — >  R”  is  an 
Euler  solution  of  (11)-(12)  for  u  G  Ad  provided  there  are  sequences  Wr  E  Par  and  er  E  O  such 
that 


•  d{'Kr)  0; 

•  sup(er)/d(7rr)  0;  and 

•  <  i->  x,rr(^;  ®£>>  u,  €r)  converges  uniformly  to  p  as  r  — »  +00. 
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We  say  that  (11)  is  ISS  for  sampling  solutions  provided  there  exist  ^  €  K.C  and  7  G  ICoo 
satisfying:  For  each  e,M,N>0  with  0  <  £  <  M,  there  exist  6  =  6{e,  M,  iV)  and  /t  =  «(£,  M,  N) 
such  that  for  each  tt  G  Par(5),  Xo  €  MBn,  u  G  and  e  G  O  for  which  sup(e)  <  /cd(7r), 

\x^{t]Xo,u,e)\  <  max{/3(M,t)  +  7(1V),£} 

for  all  t  >  0.  We  say  that  the  system  (11)  is  ISS  for  Euler  solutions  provided  there  exist  P  £  ICC 
and  7  G  ^00  satisfying:  If  u  G  At,  Xo  G  R",  and  t  ^  x{t)  is  an  Euler  solution  of  (11)-(12),  then 

|a:(t)|  < /^(koM)  +  7(|w|oo) 


for  all  t  >  0. 

Let  /i :  R”  X  R*”  — V  R"  :  (x,  u)  i->  h(x,  u)  be  continuous,  locally  Lipschitz  in  x  uniformly  on 
compact  subsets  of  R”  x  R"*,  and  so  that  h(0,0)  =  0.  Recall  that  a  control-Lyapunov  function 
(clf)  for  X  =  h{x,u)  is  a  continuous,  positive  definite,  proper  function  V  :  R”  — >  R  for  which 
there  exists  a  continuous,  positive  definite  function  W  :  R”  — >  R  and  a  nondecreasing  function 
a  :  [0, 00)  ^  [0, 00),  which  satisfy: 

VC  e  dpV{x),  min  (C,h(x,n))  <  -W(x) 

|ul<o(|x|) 

for  all  X  G  R”.  In  this  case,  one  calls  (F,  W)  a  Lyapunov  pair  for  the  system.  (The  proximal 
superdifferential  (resp.,  proximal  subdifferentiat)  of  a  function  V  :  fl  — >  R  at  x  G  fl,  where  is 
an  open  subset  of  R",  which  is  denoted  by  d^V{x)  (resp.,  5pV'(x)),  is  defined  to  be  the  set  of 
all  C  e  R”  for  which  there  exist  <r,7?  >  0  such  that  V{y)  -  V{x)  -  a\y—  xp  <  (C,y  -  x)  (resp. 
V(y)  _  v’(x)  -  (T|y  -  xp  >{C,y-x))  for  all  y  G  x  +  We  proved  in  1982  that  elf’s  always 
exist  for  GAC  systems.  This  theorem  was  considerably  refined  by  Rifford,  who  showed  that, 
for  any  GAC  system,  there  exists  a  clf  V  which  is  semiconcave  on  R”  \  {0}  so  that 

VC  e  diVix),  min  (C,/i(x,n))  <  -V{x)  (13) 

|u|<a(|i|) 

for  all  X  G  R”.  (A  continuous  function  y  :  — >  R  is  semiconcave  provided  for  any  point  Xo  €  fl, 
there  exist  p,C  >0  such  that  g{x)  +  y(y)  —  2y  <  Cljx  —  y|p  for  all  x, y  G  Xo  +  pBn-  The 

limiting  subdifferential  of  a  continuous  function  V  :f2— >RatxGf2is  5iF(x)  :=  {y  G  R”  : 
3x„  -+  X  &  y„  G  dpV{xn)  s.t.  y„  q}.  If  V  is  semiconcave,  then  it  is  locally  Lipschitz,  and 
0  diVix)  C  d^V{x)  for  all  x  G  11.)  The  construction  of  the  feedback  is  done  starting  from  a 
clf  as  in  this  result. 

Let  X  i-»  C(®)  be  any  selection  of  5lV'(x)  on  R"  \  {0}  and  C(0)  €  R"  be  arbitrary.  For  each 
X  G  R"  \  {0},  we  can  choose  u  =  Ux  €  a{\x\)Bm  that  satisfies  the  inequaUty  in  (13)  for  the 
dynamics  h{x,u)  =  f{x)  +  G{x)u  and  C  =  ((x)-  First  define  the  feedback  Ai  :  R"  — >  R"’'  by 
Ki(x)  =  Ua;  for  all  X  7^  0  and  Ki(0)  =  0.  Next,  introduce  the  following  functions:  a(x)  = 
(C(x),  /(x)  +  G(x)Ki(x)),  bj(x)  =  (((x),gj(x))  for  all  j  =  1, 2, . . . ,  m,  and  finally 

A2(x)  =  -2y(x)(sign{6i(x)},sign{62(a:)},---,sign{6,n(a:)})^, 

where  gj  is  the  jth  column  of  G,  and  where  sign{s}  =  1,  —1, 0  if  s  >  0,  <  0,  =  0  respectively. 
The  proof  then  shows  that  K  :=  Ki+  K2  is  so  that  x(t)  =  F(x(t),K(x(t)  +  e(t)),u(t))  is  ISS 
for  sampling  solutions.  An  analogous  result  holds  for  Euler  solutions. 
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Minimum-Phase  Properties 

Another  recent  area  of  research  dealt  with  notions  associated  to  the  minimum-phase  prop¬ 
erty;  we  next  describe  this  work,  done  in  collaboration  with  Liberzon  and  Morse,  cf.  [s50].  Recall 
that  a  continuous-time  linear  single-input /single-output  (SISO)  system  is  said  to  be  minimum- 
phase  if  the  numerator  polynomial  of  its  transfer  function  has  all  its  zeros  in  the  open  left  half 
plane.  If  a  linear  system  of  relative  degree  r  is  minimum-phase,  then  the  “inverse”  system, 
driven  by  the  r-th  derivative  of  the  output  of  the  original  system,  is  stable.  (For  left-invertible, 
multi-input/multi-output  (MIMO)  systems,  in  place  of  the  zeros  of  the  numerator  one  appeals 
to  the  so-called  transmission  zeros.)  The  notion  of  a  minimum-phase  system  is  of  great  sig¬ 
nificance  in  many  areas  of  linear  system  analysis  and  design.  In  particular,  it  has  played  an 
important  role  in  parameter  adaptive  control.  A  basic  example  is  provided  by  the  “certainty 
equivalence  output  stabilization  theorem” ,  which  says  that  when  a  certainty  equivalence,  out¬ 
put  stabilizing  adaptive  controller  is  applied  to  a  minimum-phase  linear  system,  the  closed-loop 
system  is  detectable  through  the  tuning  error.  In  essence,  this  result  serves  as  a  justification 
for  the  certadnty  equivalence  approach  to  adaptive  control  of  minimum-phase  linear  systems. 

For  nonlinear  systems  that  are  affine  in  controls,  a  major  contribution  of  Byrnes  and  Isidori 
was  to  define  the  minimum-phase  property  in  terms  of  the  new  concept  of  zero  dynamics.  The 
zero  dynamics  are  the  internal  dynamics  of  the  system  under  the  action  of  an  input  that  holds 
the  output  constantly  at  zero.  The  system  is  called  minimum-phase  if  the  zero  dynamics  are 
(globally)  asymptotically  stable.  In  the  SISO  case,  a  unique  input  capable  of  producing  the 
zero  output  is  guaranteed  to  exist  if  the  system  has  a  uniform  relative  degree,  which  is  now 
defined  to  be  the  number  of  times  one  has  to  difl[^erentiate  the  output  until  the  input  appears. 
(Extensions  to  MIMO  systems  are  discussed  in  Isidori’s  book  as  well.)  In  view  of  the  need  to 
work  with  the  zero  dynamics,  this  definition  of  a  minimum-phase  nonlinear  system  prompts 
one  to  look  for  a  change  of  coordinates  that  transforms  the  system  into  a  certain  normal  form. 
It  has  also  been  recognized  that  just  asymptotic  stability  of  the  zero  dynamics  is  sometimes 
insufficient  for  control  design  purposes,  so  that  additional  requirements  need  to  be  placed  on  the 
internal  dynamics  of  the  system.  One  such  common  requirement  is  that  the  internal  dynamics 
be  ISS  with  respect  to  the  output  and  its  derivatives  up  to  order  r  -  1,  where  r  is  the  relative 
degree.  Thus,  it  is  of  interest  to  develop  alternative  (and  possibly  stronger)  concepts  which  can 
be  applied  when  asymptotic  stability  of  zero  dynamics  is  difficult  to  verify  or  inadequate. 

In  the  paper  [s50],  we  introduced  the  notion  of  output-input  stability  (OIS),  which  does 
not  rely  on  zero  dynamics  or  normal  forms.  Loosely  speaking,  a  system  is  OIS  if  its  state 
and  input  eventually  become  small  when  the  output  and  derivatives  of  the  output  are  small. 
The  class  of  OIS  systems  includes  all  left-invertible  linear  systems  whose  transmission  zeros 
have  negative  real  parts  and  all  affine  systems  in  global  normal  form  with  input-to-state  stable 
internal  dynamics.  Conceptually,  the  new  notion  relates  to  the  existing  concept  of  a  minimum- 
phase  nonlinear  system  in  much  the  same  way  as  input-to-state  stability  (ISS)  relates  to  global 
asymptotic  stability  under  zero  inputs  (0-GAS),  modulo  the  duality  between  inputs  and  outputs. 
An  important  outcome  of  this  parallelism  is  that  the  tools  that  have  been  developed  for  studying 
ISS  and  related  concepts  can  be  employed  to  study  output-input  stability.  If  a  system  has  a 
uniform  relative  degree  (in  an  appropriate  sense)  and  is  detectable  through  the  output  and  its 
derivatives  up  to  some  order,  uniformly  over  all  inputs  that  produce  a  given  output,  then  it 
is  output-input  stable.  For  SISO  systems  that  are  real  analytic  in  controls,  we  also  showed 
that  the  converse  is  true,  thus  arriving  at  a  useful  equivalent  characterization  of  output-input 
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stability.  For  general  OIS  systems  in  our  sense,  we  show  in  the  paper  that  stabilizing  the  output 
and  its  derivatives  results  in  internal  stability,  as  one  should  expect  of  a  notion  of  minimum 
phase  systems.  We  also  established  a  natural  nonlinear  counterpart  of  the  certainty  equivalence 
output  stabilization  theorem  from  linear  adaptive  control,  an  intuitively  appealing  result  that 
did  not  seem  to  be  attainable  within  the  boundaries  of  the  existing  theory  of  minimum-phase 
nonlinear  systems.  Let  us  provide  a  few  technical  details.  (We  restrict  to  the  single-input 
single-output  case;  the  vector  case  is  similar  but  sHghtly  more  complicated  technically.) 

A  smooth  system  x  =  f(x,  u),  y  =  h{x),  is  defined  to  be  OIS  if  there  exist  a  positive  integer 
N,  a  class  ICC  function  13,  and  a  class  K-oo  function  7  such  that  for  every  initial  state  a;(0)  and 
every  N  —  1  times  continuously  differentiable  input  u  the  inequality 

|u(t)|-h|x(t)|  <  ;3(la;(0)|,t)-h7(l|y'^llM) 

holds  for  all  t  in  the  domain  of  the  definition  of  the  solution,  where  “3/^”  lists  y  as  well  as  its 
first  N  derivatives,  and  we  are  taking  supremum  norms  as  usual. 

For  each  nonnegative  integer  k,  restricting  the  input  u  to  be  of  class  0*'“^,  we  can  con¬ 
sider  the  k-output  extension  of  the  system,  x  =  f{x,u),  y^  =  hk{x,u, . . .  where 

:=  {Hoix);Hi{x,u);...-,Hk{x,u,...,u'^'^-'^'>))  is  the  new  output  map.  (For 
A:  =  0, 1, . . .  define,  recursively,  the  functions  Hk  :  M”  x  E*'  — »  M  by  the  formulas  Ho  :=  h  and 
Hk+iix,  uo,  . . . ,  Uk)  :=  ^fix,  uo)  +  HjZo  These  give  the  derivatives  of  the  output: 

if  the  input  u(-)  is  in  then  along  each  solution  x(  )  the  corresponding  output  has  a  con¬ 
tinuous  k-th  derivative  satisfying  =  Hk{x{t),u{t),  ^5(f)).)  In  other  words,  we 

redefine  the  output  of  the  system  to  be  y’^.  We  say  that  the  original  system  is  weakly  OSS  of 
order  k  if  its  fe-output  extension  is  OSS,  and  weakly  uniformly  OSS  of  order  k  if  its  A:-output 
extension  is  uniformly  OSS,  in  the  by  now  standard  sense  of  OSS  as  a  special  case  of  lOSS,  and 
uniformity  understood  with  respect  to  all  inputs,  or  equivalently  in  a  differential  inclusion  sense. 
We  say  that  a  positive  integer  r  is  the  (uniform)  relative  degree  of  the  system  if  the  following 
two  conditions  hold:  (1)  for  each  k  <  r,  the  function  Hk  is  independent  of  uq,  . . .  and 

(2)  there  exist  two  class  /Coo  functions  pi  and  p2  such  that  |uo|  <  pi(|a:|)  +  p2{\IIr{x,uo)\)  for 
aU  X  €  R"  and  all  uq  e  R.  (If  there  exists  such  an  integer  r,  then  it  is  imique.)  A  system  is 
defined  to  be  strongly  minimum-phase  if  it  has  a  relative  degree  r  and  is  weakly  uniformly  OSS 
of  order  r  —  1 

It  is  shown  in  [s50]  that:  (1)  given  any  system  with  relative  degree  r  and  weakly  uniformly 
OSS  of  order  k,  for  some  k,  the  system  is  OIS,  with  N  =  max{r,  k};  (2)  conversely,  if  the 
system  is  OIS,  then  it  is  weakly  uniformly  OSS  order  N,  and  if  in  addition  the  function  f{x,-) 
is  real  anal3rtic  in  u  for  each  fixed  x,  /(0, 0)  =  0  and  h{Q)  =  0,  then  it  has  a  relative  degree 
r  <  N.  Thus,  for  systems  with  relative  degree,  OIS  is  equivalent  to  weak  uniform  OSS  of  order 
k  for  some  k  <  N,  and  under  the  additional  assiunptions  on  /  and  h,  OIS  is  equivalent  to  the 
existence  of  a  relative  degree  r  <  N  plus  weak  uniform  OSS  of  order  k  for  some  k  <  N. 

2.2  Some  Details:  Systems/ Control  Problems  Inspired  by  Molecular  Biology 

We  describe  next  some  control  and  systems  problems  motivated  by  molecular  biology. 
Monotone  Systems  and  Cauchy  Gains 

In  [s31],  we  consider  systems  that  evolve  in  state  spaces  X  which  are  subsets  of  Euclidean 
space  R"  with  an  order  >-  induced  by  a  closed  convex  cone  K.  Similarly,  the  input-value  set  H 
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and  output-value  set  y  are  subsets  of  ordered  Euclidean  spaces,  with  orders  induced  by  cones 
and  Ry,  but  we  use  the  same  symbol  (h)  to  denote  any  of  the  orders.  We  say  that  a  system 
X  =  /(x,  u),  y  =  h{x)  is  monotone  provided  that  h  :  X  y  is  &  monotone  map  and  that  the 
flow  preserves  order,  i.e.  the  implication  below  holds  for  all  t  >  0: 

Ul  y  U2,  XiyX2  </>(t,Xl,«l)  h  ^{t,X2,U2) 

where  ui  'y  U2  mans  that  u\{t)  h  U2(<)  for  all  t  and  where  is  the  state  at  time  t 

obtained  if  the  initial  state  is  ^  and  the  external  input  is  u(  ).  The  map  /  is  deflned  onX  xU, 
where  X  is  some  open  subset  of  R”  which  contains  X,  and  f{x,u)  is  continuous  in  (x,  u)  and 
locally  Lipschitz  continuous  in  x  locally  uniformly  on  u.  Moreover,  we  assume  that  solutions 
with  initial  states  in  X  are  defined  for  alH  >  0  (forward  completeness)  and  that  the  set  X  is 
forward  invariant.  For  simplicity  in  this  discussion,  we  will  assume  that  the  sets  X  and  U  are 
convex,  and  that  the  order  cones  all  have  nonempty  interiors,  although  far  less  is  needed  for 
the  results  to  be  quoted  to  hold,  see  [s31].  Recall  that  the  Bouligand  (or  “contingent”)  tangent 
cone  to  a  set  <S  at  a  point  ^  is  the  set 

T^S  :=  (lim^(4i-0 

where  ->C  means  that  ^  as  z  oo  and  that  e  S  for  all  i.  The  following  is  an 
infinitesimal  characterization  of  monotonicity,  expressed  directly  in  terms  of  the  vector  field, 
which  does  not  require  the  explicit  computation  of  solutions:  A  system  is  monotone  if  and  only 
if,  for  all  ^1,^2: 

6  h  6  and  Ul  V  U2  /(6. «i)  -  /(6i  “2)  € 

(or,  equivalently,  —  ^2  G  dK  and  ui  ^  U2  /(^i)“i)  “  G  particular 

case  is  when  K  =  R>o  and  AT"  =  R>o  (with  U  =  M*").  Such  systems  are  called  cooperative 
systems.  Assuming  that  /  is  continuously  differentiable,  cooperativity  if  equivalent  to  the 
two  properties  ^•(x,u)  >  0  for  all  i  ^  j  and  ^(x,u)  >  0  for  all  i,j,  holding  for  all  x,u. 
More  general  orthants  must  be  considered,  however,  in  order  to  study  applications  like  MAPK 
cascades,  discussed  earlier.  For  such  more  general  orthants,  similar  characterizations  are  given 
in  [s31]  in  [s3],  together  with  simple  graph-theoretic  tests.  Cascades 

XI 

X2 

Xn  —  fNi^N,  w) 

of  monotone  systems  are  again  monotone  (with  respect  to  the  obvious  orders),  which  makes 
the  notion  particularly  suitable  for  decomposition  approaches  to  systems  analysis  and  control 
design. 

A  notion  of  “Cauchy  gain”  was  introduced  in  [s61]  in  order  to  quantify  the  amplification 
of  signals  by  systems  in  a  manner  useful  for  biological  applications.  For  monotone  dynamical 
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systems  satisfying  an  additional  property,  it  is  possible  to  obtain  tight  estimates  of  Cauchy 
gains.  This  is  achieved  by  showing  that  the  output  values  y{t)  corresponding  to  an  input  u(  ) 
axe  always  “sandwiched”  in  between  the  outputs  corresponding  to  two  constant  inputs  which 
bound  the  range  of  u(-)-  This  additional  property  motivated  our  looking  at  monotone  systems 
to  start  with.  We  say  that  a  system  has  a  well-defined  static  Input/State  (I/S)  characteristic 

if  for  each  constant  input  u{t)  =  u  there  exists  a  (necessarily  unique)  globally  asymptoti¬ 
cally  stable  equilibrium  fcx(u).  We  say  that  the  characteristic  is  nondegenerate  if  the  Jacobian 
Dxf(kx{u),u)  is  nonsingular,  for  all  u.  If  a  characteristic  exists,  we  also  define  the  static  In¬ 
put/Output  (I/O)  characteristic  as  ky(u)  :=  h{kx{u)).  Cascades  of  systems  with  well-defined 
I/O  characteristics  also  have  well-defined  I/O  characteristics. 

The  key  technical  property  for  monotone  systems  with  well-defined  characteristics  is  as 
follows  (there  is  also  a  multivariable  version,  but  we  provide  only  the  scalar  statement  here): 
Suppose  that  a  monotone  single-input  single-output  (“SISO”)  system  has  a  static  I/O  char¬ 
acteristic  ky{-).  Then,  for  each  initial  condition  ^  and  each  bounded  input  u(  ),  the  following 
holds: 

ky(uin{)  <  liminf  j/(t,C,u)  <  limsupj/(t,fyu)  <  ky{usap) . 

"  t-*+oo  t->+oo 

(If,  instead,  outputs  are  ordered  by  >,  then  the  inequalities  get  reversed.)  The  Cauchy  gain,  in 
the  sense  of  [s61],  can  be  estimated  firom  this  result.  For  any  signal  u  defined  on  [0,  -|-oo)  and 
taking  values  in  a  Banach  space,  we  defined  in  [s61]  its  “asymptotic  amplitude  as  follows. 

Ikllaa  :=  limsup  ||a;(t)  -  w(s)||  €  [0,oo] . 

>00 

This  measures  how  “non-Cauchy”  a  function  is,  since  ||^||aa  ~  ^  only  if  limt^oo  ^(J^)  exists. 

In  these  terms,  we  defined  a  system  as  having  a  Cauchy  gain  7  G  /Coo  if  lll/llaa  ^  7(ll«ll  aa) 
all  input/output  pairs  u,y. 

A  small-gain  theorem  for  monotone  systems  is  as  follows.  Consider  the  following  intercon¬ 
nection  of  two  SISO  dynamical  systems 

X  =  fx{x,w),  y  =  hx{x) 

Z  -  fziz,y),  w  =  hz{z) 

with  Ux  =  34  and  14  =  34.  Suppose  that:  (1)  the  first  system  is  monotone  when  its  input 
w  as  well  as  output  y  are  ordered  according  to  the  “standard  order”  induced  by  the  positive 
real  semi-axis;  (2)  the  second  system  is  monotone  when  its  input  y  is  ordered  according  to 
the  standard  order  induced  by  the  positive  real  semi-axis  and  its  output  w  is  ordered  by  the 
opposite  order  (i.e.,  the  one  induced  by  the  negative  real  semi-axis);  (3)  the  respective  static 
I/S  characteristics  fcx(  )  and  kz{-)  exist  (thus,  the  static  I/O  characteristics  ky{-)  and  ky,{-) 
exist  too  and  are  respectively  monotonically  increasing  and  monotonically  decreasing);  (4) 
every  solution  of  the  closed-loop  system  is  bounded;  and  (5)  the  scalar  discrete  time  dynamical 
system,  evolving  in  Ux'. 

Ufc+i  =  {^knj  o  ky)  (uk) 

has  a  unique  globally  attractive  equilibrium  u.  Then,  the  closed-loop  system  has  a  globally 
attractive  equilibriiun. 
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This  small-gain  result  allows  testing  stability  of  systems  of  high  dimension,  under  negative 
feedback,  by  restricting  analysis  to  a  one-dimensional  discrete-time  system.  As  a  concrete 
illustration,  let  us  consider  the  following  open-loop  system: 


V2  (100  -  xi)  _  gixi  g2  +  u 
k2  +  (100  -  xi)  ki  +  xig4  +  u 

ve  (300  -yi-ya)  _  >C3  (100  -  a:i)  yi 
fee  -b  (300  -  yi  -  ys)  ks  -f-  yi 

K4  (100  -  xi)  (300  -  yi  -  yz)  _  usys  qqa 

ki  +  (300  -  yi  -  yz)  ks  +  yz 
uio  (300  -  zi  -  zz)  _  «7  ys 
kio  +  (300  —  zi  -  Zz)  fc?  + 

«8  yz  (300  -  zi-  Zz)  _  vqZz 
ks  +  (300  -  zi-  Zz)  fcg  +  Zz 

This  is  the  model  studied  by  Kholodenko,  for  MAPK  cascades,  from  which  we  also  borrow 
the  values  of  constants  (with  a  couple  of  exceptions,  see  below):  gi  =  0.22,  g2  =  45,  g^  =  50, 
ki  =  10,  V2  =  0.25,  k2  =  8,  K3  =  0.025,  kz  —  15,  /C4  =  0.025  k4  =  15,  vz  =  0.75,  kz  =  15, 
V6  =  0.75,  kz  =  15,  K7  =  0.025,  kj  =  15,  /eg  =  0.025,  ks  =  15,  vg  =  0.5,  kg  =  15,  uio  =  0.5, 
fejQ  =  15.  Units  are  as  follows:  concentrations  and  Michaelis  constants  (fc’s)  are  expressed  in  nM, 
catalytic  rate  constants  (k’s)  in  and  maximal  enzyme  rates  (v’s)  in  nM.s~^.  Kholodenko 
showed  that  oscillations  may  arise  in  this  system  for  appropriate  values  of  negative  feedback 
gains.  (We  have  slightly  changed  the  input  term,  using  coefficients  gi,  g2,  g4,  because  we  wish 
to  emphasize  the  open-loop  system  before  considering  the  effect  of  negative  feedback.) 

To  apply  our  small-gain  theorem,  we  must  verify  that  the  system  is  monotone  and  has 
a  well-defined  I/S  characteristic.  As  a  cascade  of  a  three  systems,  of  dimensions  1,  2,  and  2 
respectively  (the  x,  y,  and  z  subsystems)  it  is  enough  to  show  these  properties  for  each  individual 
subsystem.  The  x  subsystem  is  of  dimension  one  and  easy  to  study,  so  we  concentrate  on  the 
y  and  z  subsystems.  Each  of  this  has  the  following  generic  form: 


XI  = 

yi  = 
ya  = 

ii  = 

23  = 


X^  =  —u9\{x^)  -b  ^2(0  —  x^  —  x^)  1 

X^  =  u0z{o,  —  X^  —  X^)  —  04{x^)  J 


f{x^,x^,u) 


for  an  appropriate  constant  a  (where  the  functions  have  the  form  evolving 

on  the  triangle  X  =  >  0,  >  0, +  x^  <  a}.  The  paper  [s31]  shows  that 

a  system  of  this  form  is  monotone  with  respect  to  a  suitable  orthant  order,  and  that  unique 
global  asymptotically  stable  equilibria  exist  for  each  constant  input  u.  Since  the  complete 
system  is  a  cascade  of  such  elementary  subsystems,  we  know  that  otir  small-gain  result  may 
be  applied.  Figure  1  shows  the  I/O  characteristic  of  this  system,  as  well  as  the  characteristic 
corresponding  to  a  feedback  u  =  with  the  gain  K  =  30000.  It  is  evident  from  this  planar 
plot  that  the  small-gain  condition  is  satisfied  -  a  “spiderweb”  diagram  shows  convergence.  Our 
theorem  then  guarantees  global  attraction  to  a  unique  equilibrium.  Indeed,  Figure  2  shows 
a  typical  state  trajectory.  A  somewhat  different  small-gain  theorem,  based  only  on  Cauchy 
gains  and  not  monotonicity,  was  proved  in  [s61],  and  applied  in  particular  to  the  model  of 
MAPK  cascades  under  negative  feedback  with  parameters  given  in  Shvartsman’s  work.  In  that 
reference,  a  bifurcation  analysis  is  performed  with  respect  to  the  feedback  gain  fc  as  a  parameter. 
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Figure  1:  I/O  characteristic  and  small-gain  illustration  for  MAPK  example 


Figure  2:  Simulation  of  MAPK  system  under  negative  feedback  satisfying  small-gain  conditions. 
Key:  xi  dots,  X2  dashes,  0:3  dash-dot,  X4  circles,  0:5  solid 


Numerically,  it  is  found  that  oscillations  result  when  this  parameter  attains  the  value  k  =  5.2. 
The  Cauchy  gain  approach  shows  that  stability  remains  imtil  at  least  k  =  3.92,  which  is  a 
very  tight  estimate.  Keeping  in  mind  that  om  results  are  valid  even  under  arbitrary  delays  on 
feedback  loops,  this  is  quite  remarkable. 

Robust  Stability  and  Observers  for  Chemical  Networks 

Designers  strive  to  engineer  as  much  robustness  as  possible  into  control  systems.  However, 
few  systems  perform  acceptably  under  truly  large  variations  in  parameters.  In  biology,  in 
contrast,  there  is  often  a  very  large  variability  in  intracellular  concentrations  of  chemicals,  due 
to,  for  instance,  unequal  division  among  daughter  cells  during  mitosis,  gene  duplications,  or 
mutations.  If  functions  critical  to  the  survival  of  the  organism  are  not  affected,  this  means  that 
evolution  must  have  selected  for  appropriately  robust  structures.  Thus,  the  study  of  biological 
models  might  provide  a  guide  to  novel  robust  structures  for  engineering  applications. 

The  paper  [s66],  motivated  originally  by  a  model  in  immunology,  represents  another  direc¬ 
tion  of  recent  research  by  the  PI.  It  led  to  the  study  of  classes  of  systems  for  which  stability 
is  robust  independently  of  numerical  values  of  parameters.  The  class  of  systems  studied  there 
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can  be  defined  mathematically  as  consisting  of  systems  of  the  following  form: 

i  =  f{x)  =  £  £  Oi,-  . . .  x^  {bi  -  bj)  (16) 

i=l 

where  the  constants  Oy  are  all  >  0,  and  the  matrix  A  =  [ay]  is  irreducible  (a  block  irreducibility 
assumption  is  sufficient  for  most  of  the  results).  The  number  m  can  be  thought  of  as  the  number 
of  reactions  in  a  chemical  system,  and  is  <  n,  the  dimension  of  the  system  (number  of  “species” 
participating  in  the  re2ictions).  The  column  vectors  bj  e  R"  have  entries  bij,b2j,.--, bnj,  which 
are  nonnegative  integers,  and  the  matrix  B  :=  [6i,  621  •  •  •  >  ^m]  has  rank  m,  and  none  of  its  rows 
vanishes  This  is  viewed  as  a  system  evolving  on  the  positive  orthant  R"  (which  is  easily  seen 
to  be  forward  invariant  for  the  dynamics).  The  space  V  :=  span{i)i  —  bj  :  i,j  =  1,. . . , m}  is  the 
stoichiometric  subspace.  Its  translates  are  invariant  under  the  motions  (since  t]  ■  {h  -  bj)  =  0 
implies  77  •  x{t)  =  0  for  any  vector  77). 

This  type  of  system  represents  the  “zero  deficiency  and  weakly  reversible”  chemical  networks 
studied  by  Feinberg,  Horn,  and  Jackson  in  the  1970s.  In  their  language,  we  are  dealing  with  ideal 
mass-action  kinetics,  weakly  reversible  networks,  with  deficiency  zero.  (The  latter  constraint 
mpans  that  m-i-d  =  0,  where  d  is  the  dimension  of  the  stoichiometric  subspace  and  £  is  the 
number  of  linkage  classes,  i.e.,  the  number  of  connected  components  in  the  reaction  graph.  The 
irreducibility  assumption  corresponds  to  setting  ^  =  1  for  simplicity.)  We  assume,  in  addition, 
that  there  are  no  boundary  equilibria  on  positive  classes,  that  is,  if  a  parallel  translate  p  +  D 
of  X>  is  such  that  it  intersects  the  strictly  positive  main  orthant,  then  there  are  no  equilibria  in 
the  intersection  oip  +  V  and  the  boundary  of  the  positive  main  orthant.  This  is  a  condition 
that  is  satisfied  in  most  examples  that  we  have  encountered  so  far,  and  it  is  key  in  allowing  us 
to  obtain  global  stability  statements  in  our  work.  As  an  illustration,  the  kinetic  proofreading 
system  studied  by  McKeithan  is  of  this  form.  This  system  is  described  by  the  reactions  shown 
in  Figure  3.  The  variables  are  T{t),  the  concentration  of  T-cell  receptor  (TCR)  and  M{t),  the 


Figmre  3:  Kinetic  Proofreading  Reaction  According  to  McKeithan 


concentration  of  peptide-major  histocompatibility  complex  (MHC),  as  well  as  concentrations  of 
initial  ligand-receptor  and  intermediate  complexes  Co{t)  and  Ci{t), . . .  ,C'jv(t).  The  constants 
are  fci,  the  association  rate  for  the  reaction  producing  the  initial  ligand-receptor  complex  Co 
from  TCR/MHC,  other  reaction  rates  kij  (phosphorylation,etc.),  and  dissociation  rates 
This  system  is  interesting  even  for  the  relatively  trivial  case  N  =  0:  writing  Xi{t),  i  =  1, 2, 3  for 
the  concentrations  of  T,  M,  Co  respectively,  we  arrive  to  the  equations; 

xi  =  —  A;^xiX2 -I- A:_X3 
±2  =  -k_^XiX2  +  k_X3 

X3  =  fc^XiX2  ~  k_X3. 

where  we  wrote  A:^,  k_  instead  of  fci,  fc_io  respectively.  In  the  general  formalism  shown  above, 
these  equations  would  be  obtained  as  follows.  The  complexes  T  +  M  and  Co  give  rise  to  the 
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vectors  bi  and  62  respectively,  which  list  the  species  entering  in  the  complex  (first  and  second 
in  first  complex,  third  in  the  second  complex),  and  the  matrix  A  indicates  the  graph  of  the 
reaction: 

T  4"  bi  — 


We  then  have  that 


f{xi,X2,X3) 


{0,21^^^  {^>2  ~  ^1)  +  0,l2X^{bi  —  62)  —  k^x\x\x^ 


+  k_x  1X2X2 


McKeithan  postulated  that  the  immune  recognition  signal  is  determined  by  the  steady  states 
of  this  system,  and  showed  that  steady  states  are  unique,  under  the  simplifying  assumption 
that  all  constants  are  the  same:  ki^i  =  ki,  =  k-i.  This  motivated  the  PI  to  ask  several 
basic  theoretical  questions,  dealing  with  the  structure  of  the  set  of  steady  states,  in  general,  and 
the  analysis  of  stability  (are  there  periodic  orbits,  as  in  other  chemical  examples  like  Belousov- 
Zhabotinsky  reactions?  what  about  possible  chaotic  behavior?).  Among  the  results  shown 
in  [s66],  not  just  for  the  system  above  but  also  for  any  system  (16)  of  the  general  type  being 
considered,  are  the  fact  that  the  set  E  of  equilibria  is  an  embedded  submanifold,  and  the  state 
space  is  foliated  into  invariant  sets  (stoichiometry  classes)  which  are  transversal  to  the  manifold 
E]  moreover,  positive  steady  states  are  unique  in  each  leaf  (stoichiometry  class),  aU  trajectories 
converge  to  the  set  of  equilibria,  and,  moreover,  there  is  global  asymptotic  stability,  relative  to 
each  leaf,  of  the  positive  steady  state.  The  original  papers  of  Feinberg  et.  al.  provided  many 
of  the  basic  techniques  needed  in  order  to  develop  the  theory,  but  substantial  additional  effort 
was  needed  in  order  to  prove  the  global  statements.  In  addition,  the  paper  [s66]  provided 
estimates  of  robustness  to  unmodeled  dynamics,  and  gave  extensions  to  some  non-ideal  mass 
action  kinetics,  and  also  gave  solutions  to  a  stabilization  problem  (using  inflows  and  outflows). 

Suppose  now  that  we  include  a  measurement  function  h  :  E"  — >  into  the  specification 
of  the  system.  Specifically,  we  will  assume  that  the  output  coordinates  are  monomials,  i.e. 
h{x)  =  col  {x^^^x^^  •  •  •  . . . ,  . . .  x^)  where  the  p  x  n  matrix  of  exponents  C  =  (cij) 

has  all  its  entries  nonnegative  integers.  (This  choice  of  output  functions  allows  one  to  include 
reaction  rates  as  well  as  measurements  of  single  concentrations.)  As  usual,  we  define  an  observer 
for  any  system  x  =  /(x),  y  =  h{x)  to  be  another  system  z  =  g{z,  y),  x  =  h{z)  whidi  estimates 
internal  states  (in  this  case,  chemical  concentrations).  That  is,  we  assume  that  for  all  initial 
conditions  a;(0),2:(0)  of  the  composite  system,  solutions  are  well-defined  for  all  <  >  0,  and 
|x(i)  —  a:(t)|  — >  0  as  f  — >  -f  oo.  Even  though  we  are  dealing  with  potentially  highly  nonlinear 
systems,  in  our  work  with  Madalena  Chaves  cf.  [s54]  and  [s71],  we  gave  necessary  and  sufficient 
conditions  for  detectability,  and  an  explicit  construction  of  globally  convergent  observers.  The 
observer  has  the  following  very  simple  form: 

z  =  f{z)  +  C’{y-hiz))  (x  =  z) 

which,  of  covuse,  resembles  a  Luenberger  observer  (or  “deterministic  Kalman  filter”)  —  but 
which  has  the  transpose  of  C  as  a  particular  gain  matrix.  (Interestingly,  other  observers,  for 
instance  using  gains  based  on  linearized  systems  or  on  extended  Kalman  filters,  can  be  shown 
by  coimterexample  to  not  always  result  in  global  convergence,  for  these  same  systems.)  The 
proof  is  based  on  showing  that  the  system  with  inputs  i  =  f{z)+C'{u—h{z))  is  ISS,  relative  not 
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to  “z  =  0  and  u  =  0”  but  to  suitable  equilibria;  this  is,  in  turn,  done  by  using  an  entropy-like 
ISS-Lyapunov  function.  As  a  bonus,  one  obtains  an  automatic  robustness  to  observation  noise, 
expressed  in  ISS-like  terms.  This  is  one  of  the  few  instances  where  one  can  obtmn  globally 
convergent  observers  for  a  nontrivial  class  of  nonlinear  systems  (not  linearizable  by  output 
injection).  It  remains  to  see  if  such  observers  are  useful  in  interesting  engineering  applications, 
but  we  did  perform  experiments  (described  in  Chaves’  recently  completed  Ph.D.  thesis  under 
the  P.I.’s  direction)  vahdating  the  use  of  these  observers  in  some  simple  chemical  reactions. 
(We  used  NMR  data  to  actually  measure  all  concentrations,  thus  providing  a  cross-check  for 
the  estimates.)  In  [s69],  we  introduced  a  variant,  based  on  log-barrier  functions,  that  trades 
slower  observer  convergence  for  a  larger  margin  of  robustness. 

Internal  Model  Principle 

As  discussed  earlier,  in  [s41]  we  provided  an  internal  model  theorem  (in  the  sense  of  proving 
necessity  of  internal  models)  which  is  suitable  for  applications  in  which  the  goal  is  adaptation 
(disturbance  rejection)  with  signal  detection.  Let  us  give  some  details  now.  We  dealt  with 
single-input  single-output  systems  E,  affine  in  inputs:  a:(t)  =  f{x{t))  -|-  u{t)g{x{t)),  y{t)  = 
h{x{t))  with  /  and  g  smooth  vector  fields  and  h  a  scalar  smooth  function,  satisfying  /(O)  = 
/i(0)  =  0.  Given  a  class  U  of  functions  [0,  oo)  — >  K  (such  as  for  example  the  set  of  all  constant 
functions),  we  say  that  E  adapts  to  inputs  in  U  (a  more  appropriate  technical  control-theoretic 
term  woidd  be  “asymptotically  rejects  disturbances  in  W)  if  the  following  property  holds:  for 
each  u  E  U  and  each  initial  state,  the  solution  exists  for  all  t  >  0  and  is  bounded,  and  the 
corresponding  output  y{t)  =  h{x{t))  converges  to  zero  as  t  oo.  We  say  that  E  contains  an 
output-driven  internal  model  ofU  if  there  is  a  change  of  coordinates  which  brings  the  system 
into  the  following  block  form: 

ii  =  h{^u^2)-^ugi{zuZ2) 

Z2  =  f2iy,Z2) 

y  ==  f^{zi) 

(the  subsystems  with  variables  zi  and  Z2  correspond  respectively  to  So  and  Sjm  in  Figure  4), 
and  in  addition  the  subsystem  with  state  variables  Z2  is  capable  of  generating  all  functions 


w(*)  E  W 


y(^) 


0 


Figure  4:  Decomposition  of  S  into  So  and  Sim,  Latter  Driven  by  y{t) 


in  ZY,  meaning  the  following  property:  there  is  some  scalar  function  <p{z2)  so  that,  for  each 
possible  u  there  is  some  solution  of  Z2  =  /2(0, 2:2)  which  satisfies  (p{z2{t))  ^  u{i).  The 
precise  meaning  of  “change  of  coordinates”  is  as  follows.  There  must  exist  an  integer  r  <  n, 
differentiable  manifolds  Zi  and  Z2  of  dimensions  r  and  n  -  r  respectively,  a  smooth  function 
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K  :  Zi  —*  M.,  vector  fields  F  and  G  on  x  Z2  which  take  the  partitioned  form 


F  = 


f  fl{zi,Z2)  \ 
\f2iK{Zl),Z2))  ’ 


and  a  diffeomorphism  $  ;  R"  — >  x  Z2,  such  that 

{x)f{x)  =  F($  (x)) ,  {x)g{x)  =  G($(x)) ,  k($i  (x))  =  h{x) 


for  all  X  £  R",  where  $1  is  the  Zi-component  of  ^  and  star  indicates  Jacobian.  Our  result 
holds  under  additional  conditions  on  the  vector  fields  defining  the  system.  The  first  condition 
is  the  fundamental  one  from  an  intuitive  point  of  view,  namely  that  the  system  is  able  to  detect 
changes  in  the  input  signal:  Assumption  1:  a  uniform  relative  degree  exists.  This  means  that 
there  exists  some  positive  integer  r  such  that  LgL^h  =  0  V fc  <  r  —  1  and  LgLj  h{x)  ^  0  Vx  £ 
R"  where  Lx  h  is  the  Lie  derivative  of  a  function  h  along  the  direction  of  the  vector  field  X. 
(The  integer  r  is  the  relative  degree  of  S;  the  assumption  amounts  to  the  statement  that  the 
output  derivatives  must  be  independent  of  the  value  of  the  input  at  time  t,  for  all  A:  <  r, 

but  that  =  b{x{t))  +  a{x{t))u{t)  for  some  function  a(x)  which  is  everywhere  nonzero.) 

The  next  two  conditions  are  of  a  technical  nature.  They  are  automatically  satisfied  for  linear 
systems.  For  nonlinear  systems,  we  need  such  conditions  in  order  to  guarantee  the  existence 
of  a  change  of  variables  exhibiting  the  system  Sim-  (See  the  paper  for  ways  of  weakening 
these  assumptions.)  Assuming  that  the  degree  is  r,  we  introduce  the  following  vector  fields: 
5(x)  =  m  =  fix)  -  {L^fh{x))g{x),  n  :=  ady%i  =  l,...r,  where  adx  is 

the  operator  adx^  =  [X,Y]  =  (Lie  bracket).  The  assumptions  are  then:  Assumption  2:  is 
complete,  for  i  =  1, . . .  ,r,  and  Assumption  3:  the  vector  fields  n  commute  with  each  other. 
Finally,  we  must  define  the  allowed  classes  of  inputs  li.  As  usual  in  control  theory,  we  assumed 
that  inputs  axe  generated  by  exosystems.  That  is,  there  is  given  a  system  T:  w  =  Q{w),  u  —  6{w) 
evolving  on  some  differentiable  manifold,  Q  a  smooth  vector  field,  and  6  a  real- valued  smooth 
function  such  that  the  input  class  U  consists  exactly  of  those  inputs  u{t)  =  d{w{t)),  t  >  0,  for 
all  possible  solutions  of  ly  =  Q{w).  (For  example,  if  we  are  interested  in  constant  signals,  we 
pick  w  =  0,u  —  w  and  if  we  are  interested  in  sinusoidals  with  frequency  u  then  we  use  xi  =  0:2, 
^2  =  — u  =  xi.)  We  assumed  that  the  exosystem  is  Poisson-stable.  This  means  that  the 
exosystem  is  almost-periodic  in  the  sense  that  trajectories  keep  returning  to  neighborhoods  of 
the  initial  state.  We  then  proved  that,  if  Assumptions  1-3  hold  and  the  system  E  adapts  to 
inputs  in  a  class  ti  generated  by  a  Poisson-stable  exosystem,  then  S  contains  an  output-driven 
internal  model  of  U. 
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